N-gram Research Articles

Detecting Dengue/Flu Infections Based on Tweets Using LSTM and Word Embedding

With the massive spike in the use of Online Social Network Sites (OSNSs) platforms such as Web 2.0, microblogs services and online blogs, etc., valuable information in the form of sentiment, thoughts, opinions, as well as epidemic outbreaks, etc. are transferred. With the OSNSs being widely accessible, this work aims at proposing a novel approach for disease (dengue or flu) detection based on social media posts. For this purpose, an automated approach is designed with the help of LSTM (Long Short Term Memory) and word embedding techniques. Then the performance of the proposed approach is validated using a set of standard evaluation matrices. In addition, the effectiveness of the selected models is evaluated with performance measurement techniques. The accuracy of the proposed research approach is evaluated using two word embedding techniques; Word2Vec with Skip-gram (SG) and Word2Vec with Continuous-bag-of-words (CBOW). Based on the results conducted in this paper the LSTM Word2Vec with CBOW achieved better results compared to LSTM with Word2Vec SG features embedding technique. Our findings prove that the proposed method yields 94% accuracy compared to state-of-the-art approaches. Consequently, LSTM performed better than other leading methods in the detection of disease-infected people in tweets. In the end, spatial analysis is performed to identify the disease infected region.

Sense disambiguation for Punjabi language using supervised machine learning techniques

Automatic identification of a meaning of a word in a context is termed as Word Sense Disambiguation (WSD). It is a vital and hard artificial intelligence problem used in several natural language processing applications like machine translation, question answering, information retrieval, etc. In this paper, an explicit WSD system for Punjabi language using supervised techniques has been analysed. The sense tagged corpus of 150 ambiguous Punjabi noun words has been manually prepared. The six supervised machine learning techniques Decision List, Decision Tree, Naive Bayes, K-Nearest Neighbour (K-NN), Random Forest and Support Vector Machines (SVM) have been investigated in this proposed work. Every classifier has used same feature space encompassing lexical (unigram, bigram, collocations, and co-occurrence) and syntactic (part of speech) count based features. The semantic features of Punjabi language have been devised from the unlabelled Punjabi Wikipedia text using word2vec continuous bag of word and skip gram shallow neural network models. Two deep learning neural network classifiers multilayer perceptron and long short term memory have also been applied for WSD of Punjabi words. The word embedding features have experimented on six classifiers for the Punjabi WSD task. It has been observed that the performance of the supervised classifiers applied for the WSD task of Punjabi language has been enhanced with the application of word embedding features. In this work, an accuracy of 84% has been achieved by LSTM classifier using word embedding feature.

Tele-Operated Android Robot Reminiscence Group Therapy and Human Coordinated RGT for Older Adults with Dementia: A Comparative Study

Background/Purpose: Since being designated a super-aged society in 2007, Japan has faced a shortage of care givers working at nursing facilities. To address this problem, the government has been promoting research into robotics technology. However, social robots in current use have limited communicative ability. We examined the effectiveness of a tele-operated android robot in reminiscence group therapy (RGT) for persons with dementia (PWD), compared to RGT with a human coordinator. Method: Six PWD from a public facility for senior residents underwent RGT using a tele-operated android robot, Telenoid R3. Participants were divided into two groups: RGT with a human coordinator (condition 1), and RGT with the same coordinator, using Telenoid R3 (condition 2). After the first three sessions, the robot and the human coordinator were substituted in each group, and three more sessions were conducted. Linguistic data comprising the number of utterances, sentence-final particles, and N-gram was collected and analyzed. Results: Among participants, half (n = 3) showed a significantly higher average number of utterances and sentence-final particles in RGT with condition 1 (p ＜ .01, p ＜ .05, p ＜ .01), while one female participant showed a significantly higher number of sentence-final particles in RGT with the Telenoid R3 (p ＜ .05). There was no between-group difference in entropy of N-grams among subjects. Conclusion: These results suggest that RGT by tele-operated robot can promote communication in a way similar to that of human-facilitated RGT. We conclude that RGT using a tele-operated robot may increase communication opportunities for people who are unable to present to nursing facilities.

Word Embeddings and Its Application in Deep Learning

Word embedding in simple term can be defined as representing text in form of vectors. Vector representations of text help people in finding similarities, because contextual words that seem to appear nearby regularly use to appear in close proximity in vector space. The motivating factor behind such numerical representation of text corpus is that it can be manipulated arithmetically just like any other vector. Deep learning along with neural network is not new at all, both the concepts are prevalent around the decades but there was a major tailback of unavailability and accessibility of computation power. Deep learning is now effectively being used in Natural Language Processing with the improvement in techniques like word embedding, mobile enablement and focus on attention. The paper will discuss about the two popular model of word embedding (Word2Vec model) can be used for deep learning and will also compare them. The implementation steps of Skip gram model are also discusses in the paper. The paper will also discuss challenging issues for Word2Vce model.

Using an analogical reasoning framework to infer language patterns for negative life events

BackgroundFeelings of depression can be caused by negative life events (NLE) such as the death of a family member, a quarrel with one’s spouse, job loss, or strong criticism from an authority figure. The automatic and accurate identification of negative life event language patterns (NLE-LP) can help identify individuals potentially in need of psychiatric services. An NLE-LP combines a person (subject) and a reasonable negative life event (action), e.g. <parent:divorce> or < boyfriend:break_up>.MethodsThis paper proposes an analogical reasoning framework which combines a word representation approach and a pattern inference method to mine/extract NLE-LPs from psychiatric consultation documents. Word representation approaches such as skip-gram (SG) and continuous bag-of-words (CBOW) are used to generate word embeddings. Pattern inference methods such as cosine similarity (COSINE) and cosine multiplication similarity (COSMUL) are used to infer patterns.ResultsExperimental results show our proposed analogical reasoning framework outperforms the traditional methods such as positive pairwise mutual information (PPMI) and hyperspace analog to language (HAL), and can effectively mine highly precise NLE-LPs based on word embeddings. CBOW with COSINE of analogical reasoning is the best word representation and inference engine. In addition, both word embeddings and the inference engine provided by the analogical reasoning framework can further be used to improve the HAL model.ConclusionsOur proposed framework is a very simple matching function based on these word representation approaches and is applied to significantly improve HAL model mining performance.

The MeSH-Gram Neural Network Model: Extending Word Embedding Vectors with MeSH Concepts for Semantic Similarity.

Eliciting semantic similarity between concepts remains a challenging task. Recent approaches founded on embedding vectors have gained in popularity as they have risen to efficiently capture semantic relationships. The underlying idea is that two words that have close meaning gather similar contexts. In this study, we propose a new neural network model, named MeSH-gram, which relies on a straightforward approach that extends the skip-gram neural network model by considering MeSH (Medical Subject Headings) descriptors instead of words. Trained on publicly available PubMed/MEDLINE corpus, MeSH-gram is evaluated on reference standards manually annotated for semantic similarity. MeSH-gram is first compared to skip-gram with vectors of size 300 and at several windows' contexts. A deeper comparison is performed with twenty existing models. All the obtained results with Spearman's rank correlations between human scores and computed similarities show that MeSH-gram (i) outperforms the skip-gram model and (ii) is comparable to the best methods that need more computation and external resources.

Building an Experimental German User Interface Terminology Linked to SNOMED CT.

We describe the process of creating a User Interface Terminology (UIT) with the goal to generate a maximum of German language interface terms that are mapped to the reference terminology SNOMED CT. The purpose is to offer a high coverage of medical jargon in order to optimise semantic annotations of clinical documents by text mining systems. The first step consisted in the creation of an n-gram table to which words and short phrases from the English SNOMED CT description table were automatically extracted and entered. The second step was to fill up the n-gram table with human and machine translations, manually enriched by POS tags. Top-down and bottom-up methods for manual terminology population were used. Grammar rules were formulated and embedded into a term generator, which then created one-to-many German variants per SNOMED CT description. Currently, the German user interface terminology contains 4,425,948 entries, created out of 111,605 German n-grams, assigned to 95,298 English n-grams. With 341,105 active concepts and 542,462 (non FSN) descriptions, it corresponds to an average of 13 interface terms per concept and 8.2 per description. An analysis of the current quality of this resource by blinded human assessment terminology states equivalence regarding term understandability compared to a fully automated Web-based translator, which, however does not yield any synonyms, so that there are good reasons to further develop this semi-automated terminology engineering method and recommend it for other language pairs.

Sentence Embedding and Convolutional Neural Network for Semantic Textual Similarity Detection in Arabic Language

The continuous increase in extraordinary textual sources on the web has facilitated the act of paraphrase. Its detection has become a challenge in different natural language processing applications (e.g., plagiarism detection, information retrieval and extraction, question answering, etc.). Different from western languages like English, few works have been addressed the problem of extrinsic paraphrase detection in Arabic language. In this context, we proposed a deep learning-based approach to indicate how original and suspect documents expressed the same meaning. Indeed, word2vec algorithm extracted the relevant features by predicting each word to its neighbors. Subsequently, averaging the obtained vectors was efficient for generating sentence vectors representations. Then, convolutional neural network was useful to capture more contextual information and compute the degree of semantic relatedness. Faced to the lack of resources publicly available, paraphrased corpus was developed using skip gram model. It had better performance in replacing an original word by its most similar one that had the same grammatical class from a vocabulary. Finally, the proposed system achieved good results enhancing an efficient contextual relationship detection between Arabic documents in terms of precision (85%) and recall (86.8%) than previous studies.

Distributional Semantics Meets Multi-Label Learning

We present a label embedding based approach to large-scale multi-label learning, drawing inspiration from ideas rooted in distributional semantics, specifically the Skip Gram Negative Sampling (SGNS) approach, widely used to learn word embeddings. Besides leading to a highly scalable model for multi-label learning, our approach highlights interesting connections between label embedding methods commonly used for multi-label learning and paragraph embedding methods commonly used for learning representations of text data. The framework easily extends to incorporating auxiliary information such as label-label correlations; this is crucial especially when many training instances are only partially annotated. To facilitate end-to-end learning, we develop a joint learning algorithm that can learn the embeddings as well as a regression model that predicts these embeddings for the new input to be annotated, via efficient gradient based methods. We demonstrate the effectiveness of our approach through an extensive set of experiments on a variety of benchmark datasets, and show that the proposed models perform favorably as compared to state-of-the-art methods for large-scale multi-label learning.

A Performance Evaluation of Attribute Extraction Technique for the Analysis of Sentiment

Sentiment Analysis is the process of computation identifying and categories the thought expressed by people in order to determine whether the writer’s attitude towards a particular product is positive or negative. The results of this analysis can be used in computing customer satisfaction metrics, marketing, contextual advertising, suggestion systems based on the user likes and rating, recommendation systems etc. It can be done on reviews, blogs, tweets, forums etc. Term Weighting is the process of assigning weights to the terms in the documents. Different term weighting schemes like Term Frequency, Term Frequency – Inverse Document Frequency and Binary Weighting methods are used on the reviews to weigh the sentiments words present in the reviews. It is done considering the terms as unigrams, bigrams, trigrams and Syntactic n-grams. The huge data presented to extract sentiments are unstructured in nature and they require processing like classification or clustering to get some meaningful information out of it. Supervised machine learning approaches like Naive Bayes are used to classify the human sentiments present in the reviews as positive and negative.

An Evaluation of Structured Language Modeling for Automatic Speech Recognition

We evaluated probabilistic lexicalized tree-insertion grammars (PLTIGs) on a classification task relevant for automatic speech recognition. The baseline is a family of n-gram models tuned with Witt ...

An anomaly detection system based on variable N-gram features and one-class SVM

Context: Run-time detection of system anomalies at the host level remains a challenging task. Existing techniques suffer from high rates of false alarms, hindering large-scale deployment of anomaly...

Detection of Sarcasm in Text Data using Deep Convolutional Neural Networks

With the ever increasing acceptance of Online Social Networks (OSNs), a new dimension has evolved for communication amongst humans. People now share continuous streams of messages to contribute their interests, indications, discuss activities, status, latest trends and much more. This gives an opportunity to monitor and mine the opinions of a large number of online active population in real time. On one side where researchers can find out a pattern to judge the mood of the user, a serious problem of detection of irony and sarcasm in textual data poses threat to the accuracy of the techniques evolved till date. In this paper, we are proposing a deep learning based convolutional neural network method that focuses on the skip gram technique to convert words to vectors and further to detect and identify the sarcasm and irony from the normal polarized text taken from Twitter. We try to discover patterns from the textual datasets based on the identifiable features. Our approach is giving an overall accuracy of 89.9%.

A Case Study on Sepsis Using PubMed and Deep Learning for Ontology Learning.

We investigate the application of distributional semantics models for facilitating unsupervised extraction of biomedical terms from unannotated corpora. Term extraction is used as the first step of an ontology learning process that aims to (semi-)automatic annotation of biomedical concepts and relations from more than 300K PubMed titles and abstracts. We experimented with both traditional distributional semantics methods such as Latent Semantic Analysis (LSA) and Latent Dirichlet Allocation (LDA) as well as the neural language models CBOW and Skip-gram from Deep Learning. The evaluation conducted concentrates on sepsis, a major life-threatening condition, and shows that Deep Learning models outperform LSA and LDA with much higher precision.

Keywords in Context (Using n-grams) with Python

Esta lección retoma los pares de frecuencias recolectados en [Contar frecuencias de palabras][] y crea una salida de datos en HTML.

Experience of Creation of the N-Gram Module of the System “Manuscript” and Evaluation of The Efficiency of Its Application to Search Collocations in the Corpus of M.V. Lomonosov

В статье описаны функции и параметры модуля n-грамм информационно-аналитической системы (корпуса) «Манускрипт» и итоги эксперимента по применению нескольких статистических методов в корпусе текстов М. В. Ломоносова. Показано, что количественные и статистические методы оценки биграмм применимы к авторскому историческому корпусу и позволяют выявлять устойчивые сочетания.

8 million books meet in science class: Learning about big data and history of biology using Google Ngram Viewer

8 million books meet in science class: Learning about big data and history of biology using Google Ngram Viewer

A Study on Word Vector Models for Representing Korean Semantic Information

This paper examines whether the Global Vector model is applicable to Korean data as a universal learning algorithm. The main purpose of this study is to compare the global vector model (GloVe) with the word2vec models such as a continuous bag-of-words (CBOW) model and a skip-gram (SG) model. For this purpose, we conducted an experiment by employing an evaluation corpus consisting of 70 target words and 819 pairs of Korean words for word similarities and analogies, respectively. Results of the word similarity task indicated that the Pearson correlation coefficients of 0.3133 as compared with the human judgement in GloVe, 0.2637 in CBOW and 0.2177 in SG. The word analogy task showed that the overall accuracy rate of 67% in semantic and syntactic relations was obtained in GloVe, 66% in CBOW and 57% in SG.

Converting System of Phonetics Transcriptions to Myanmar Text Using N-grams Language Models

Converting between Phonetics transcriptions and Myanmar text is a process of converting between the sequence of Phonetics transcriptions and Myanmar text. Phonetics transcription is based on the pronunciation of the language and the Myanmar text is based on the written language. One Phonetics alphabet can be represented many possible forms in written language that leads into word sense ambiguity problem. Another problem is that both of the Phonetics transcriptions and Myanmar text have no space to identify the boundary of syllables and words. This problem can be defined as segmentation problem for matching and mapping between Phonetics transcriptions and Myanmar text. To solve the word-sense ambiguity problem, the research developed n-grams language models from correct training data in Myanmar language. By using these trained n-grams language models, the system can be converted from Phonetics to Myanmar text. Instead of computing the probability on the trained n-grams data, the system matched the input data and the trained n-grams model data. The system has built n-grams models where unigram model, bi-grams model, trigrams model, 4-grams models and 5-grams models to train and convert between Phonetics and Myanmar text. To solve the segmentation problem, the system needed to break the input text into individual tokens. In the system, each token may be represented the consonant, or consonant clusters or vowels. To segment the input text Myanmar text or Phonetics transcriptions correctly, the proposed used the Unicode fonts for both Myanmar text and Phonetics transcriptions.

Words and N-gram Models Analysis for “A Dream of Red Mansions”

Words and N-gram Models Analysis for “A Dream of Red Mansions”