N-gram Model Research Articles

Analysis of Fine Dining Restaurant Reviews for Perception of Customers Restaurant Service Quality

The purpose of this study is to model the perception of customers’ service quality in fine dining restaurants (FDRs) and to determine customer sentiments towards the service quality. I analyzed 22,104 reviews of 25 restaurants on TripAdvisor through Aspect-Based Sentiment Analysis (ABSA). In terms of n-gram language models, the classification performance of sentiment polarity was tested with Support Vector Machine (SVM), Naive Bayes (NB), C4.5, and Gradient Boosted Trees (GBT). I compared the performance of the model with Cohen’s kappa, accuracy, precision, recall, and F-measure results. I found five topic models service, experience, surprise, taste, and food kind by using latent Dirichlet allocation (LDA). In sentiment classification, SVM achieved the best results in bigram with 74.5% average F-measure, 94.4% accuracy, and 49.2% kappa results. This study contributes to the elements related to the perception of service quality in FDRs with psychological quality proposed by the surprise topic. This is one of the few studies conducted with ABSA on the perception of service quality in FDRs, and it is the first study examining the issue in terms of n-gram language models.

170 An Integrated Voice Recognition and Natural Language Processing Platform to Automatically Extract Thoracolumbar Injury Classification Score (TLICS) Features from Radiology Reports

INTRODUCTION: Many predictive models have been reported in the literature for clinical outcomes after spine surgery. However, the real-time implementation of these predictive scores in clinical practice is limited by the time intenstive nature of manually abstracting relevant predictors from the electronic health record. Natural language processing (NLP) provides a framework to automate data abstraction for commonly used predictive scores in spine surgery. METHODS: We retrieved and annotated the radiology reports of all Mayo Clinic patients with an International Classification of Diseases(ICD)-9/10 code corresponding to a fracture of the thoracolumbar spine between January 2005 and October 2020. Annotated data was used to train an n-gram NLP model using machine learning (ML) methods including random forest(RF), stepwise linear discriminant analysis (sLDA), k-nearest neighbors (kNN), and penalized logistic regression (pLR) models. RESULTS: 1,085 spine radiology reports were included in our analysis consististing of 868 to train our ML-NLP model and 217 for validation. Our dataset included 483 compression, 401 burst, 103 translational/rotational, and 98 distraction fractures. 103 reports had documented an injury of the PLC. The overall accuracy of the RF model for fracture morphology feature detection was 76.96% versus 65.90% in the stepwise LDA, 50.69% in the kNN, and 62.67% in the pLR. The overall accuracy to detect PLC integrity was highest in the RF model at 83.41% versus 70.97% in sLDA, 70.05% in kNN, and 67.74% in pLR model. Our RF model was implemented in the backend of a web application where users can dictate reports and have TLICS score features be automatically extracted. CONCLUSION: We have developed a robust ML-NLP model for extracting TLICS score features from radiology reports, which we deployed in a convenient web application that can be integrated prospectively into clinical practice and scaled to other commonly used predictive scores.

Language Models Explain Word Reading Times Better Than Empirical Predictability.

Though there is a strong consensus that word length and frequency are the most important single-word features determining visual-orthographic access to the mental lexicon, there is less agreement as how to best capture syntactic and semantic factors. The traditional approach in cognitive reading research assumes that word predictability from sentence context is best captured by cloze completion probability (CCP) derived from human performance data. We review recent research suggesting that probabilistic language models provide deeper explanations for syntactic and semantic effects than CCP. Then we compare CCP with three probabilistic language models for predicting word viewing times in an English and a German eye tracking sample: (1) Symbolic n-gram models consolidate syntactic and semantic short-range relations by computing the probability of a word to occur, given two preceding words. (2) Topic models rely on subsymbolic representations to capture long-range semantic similarity by word co-occurrence counts in documents. (3) In recurrent neural networks (RNNs), the subsymbolic units are trained to predict the next word, given all preceding words in the sentences. To examine lexical retrieval, these models were used to predict single fixation durations and gaze durations to capture rapidly successful and standard lexical access, and total viewing time to capture late semantic integration. The linear item-level analyses showed greater correlations of all language models with all eye-movement measures than CCP. Then we examined non-linear relations between the different types of predictability and the reading times using generalized additive models. N-gram and RNN probabilities of the present word more consistently predicted reading performance compared with topic models or CCP. For the effects of last-word probability on current-word viewing times, we obtained the best results with n-gram models. Such count-based models seem to best capture short-range access that is still underway when the eyes move on to the subsequent word. The prediction-trained RNN models, in contrast, better predicted early preprocessing of the next word. In sum, our results demonstrate that the different language models account for differential cognitive processes during reading. We discuss these algorithmically concrete blueprints of lexical consolidation as theoretically deep explanations for human reading.

New Language Models for Spelling Correction

Correcting spelling errors based on the context is a fairly significant problem in Natural Language Processing (NLP) applications. The majority of the work carried out to introduce the context into the process of spelling correction uses the n-gram language models. However, these models fail in several cases to give adequate probabilities for the suggested solutions of a misspelled word in a given context. To resolve this issue, we propose two new language models inspired by stochastic language models combined with edit distance. A first phase consists in finding the words of the lexicon orthographically close to the erroneous word and a second phase consists in ranking and limiting these suggestions. We have applied the new approach to Arabic language taking into account its specificity of having strong contextual connections between distant words in a sentence. To evaluate our approach, we have developed textual data processing applications, namely the extraction of distant transition dictionaries. The correction accuracy obtained exceeds 98% for the first 10 suggestions. Our approach has the advantage of simplifying the parameters to be estimated with a higher correction accuracy compared to n-gram language models. Hence the need to use such an approach.

Covid-19 and E-Learning: An Exploratory Analysis of Research Topics and Interests in E-Learning During the Pandemic

E-learning has gained further importance and the amount of e-learning research and applications has increased exponentially during the COVID-19 pandemic. Therefore, it is critical to examine trends and interests in e-learning research and applications during the pandemic period. This paper aims to identify trends and research interests in e-learning articles related to COVID-19 pandemic. Consistent with this aim, a semantic content analysis was conducted on 3562 peer-reviewed journal articles published since the beginning of the COVID-19 pandemic, using the N-gram model and Latent Dirichlet Allocation (LDA) topic modeling approach. Findings of the study revealed the high-frequency bigrams such as “online learn”, “online education”, “online teach” and “distance learn”, as well as trigrams such as “higher education institution”, “emergency remote teach”, “education online learn” and “online teach learn”. Moreover, the LDA topic modeling analysis revealed 42 topics. The topics of “Learning Needs”, “Higher Education” and “Social Impact” respectively were the most focused topics. These topics also revealed concepts, dimensions, methods, tools, technologies, applications, measurement and evaluation models, which are the focal points of e-learning field during the pandemic. The findings of the study are expected to provide insights to researchers and future studies.

Just-In-Time Defect Identification and Localization: A Two-Phase Framework

Defect localization aims to locate buggy program elements (e.g., buggy files, methods or lines of code) based on defect symptoms, e.g., bug reports or program spectrum. However, when we receive the defect symptoms, the defect has been exposed and negative impacts have been introduced. Thus, one challenging task is: whether we can locate buggy program prior to the appearance of the defect symptom (e.g., when buggy program elements are being committed to a version control system). We refer to this type of defect localization as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">“Just-In-Time (JIT) Defect localization”</i> . Although many prior studies have proposed various JIT defect identification methods to identify whether a new change is buggy, these prior methods do not locate the suspicious positions. Thus, JIT defect localization is the next step of JIT defect identification (i.e., after a buggy change is identified, suspicious source code lines are located). To address this problem, we propose a two-phase framework, i.e., JIT defect identification and JIT defect localization. Given a new change, JIT defect identification will identify it as buggy change or clean change first. If a new change is identified as buggy, JIT defect localization will rank the source code lines introduced by the new change according to their suspiciousness scores. The source code lines ranked at the top of the list are estimated as the defect location. For JIT defect identification phase, we use 14 change-level features to build a classifier by following existing approach. For JIT defect localization phase, we propose a JIT defect localization approach that leverages software naturalness with the N-gram model. To evaluate the proposed framework, we conduct an empirical study on 14 open source projects with a total of 177,250 changes. The results show that software naturalness is effective for our JIT defect localization. Our model achieves a reasonable performance, and outperforms the two baselines (i.e., random guess and a static bug finder (i.e., PMD)) by a substantial margin in terms of four ranking measures.

A Novel Human-Vehicle Interaction Assistive Device for Arab Drivers Using Speech Recognition

About one-quarter of all car collisions in the United States are caused by distracted driving, and this ratio is expected to rise. As vehicles are equipped with more elaborate and complex technology, human-vehicle interaction via dashboard displays and controls will become more complex and distracting. Human-vehicle interaction via voice-based technology offers a less distracting alternative. In this study we aim to develop a voice-based car assistant, with a focus on Arabic language speech recognition. We prepare a new 4000-word domain-specific lexicon to comprehensively support driver-vehicle interactions, and we create corresponding text and speech corpora. Then we extract acoustic feature vectors and use various acoustic models to support speech recognition. The language model is created using an n-gram model. Then acoustic and language models, and the lexicon are combined to generate a decoding graph. The text corpus consists of 6110 elements, including words, phrases, and sentences. The speech corpus has more than 60000 recordings (almost 50 hours). For the decoding of noise-free audio, a Deep Neural Network + Hidden Markov Model provided 94.832% accuracy, a Subspace Gaussian Mixture Model + Hidden Markov Model provided 94.2% accuracy, and the best Gaussian Mixture Model + Hidden Markov Model provided 94.13% accuracy. For the decoding of noisy audio, a Deep Neural Network + Hidden Markov Model provided 93.316% accuracy, a Subspace Gaussian Mixture Model + Hidden Markov Model provided 92.62% accuracy, and the best Gaussian Mixture Model + Hidden Markov Model provided 91.82% accuracy. A usability study was conducted on the system with 10 participants. Almost all of the results of that study showed usability ratings of greater than 4.0 out of 5.0. These usability ratings indicate that the proposed system was seen by the participants as important, and useful for reducing driver distraction.

Automatic Speech Recognition for Qur’an Verses using Traditional Technique

Deep learning is the one of approaches of machine learning that uses algorithms for building a model based on complex unstructured data. The Muslims Holy Qur’an book is written using Arabic diacritized text. In this paper, a traditional method to build a robust Qur’an versus recognition is proposed. The MFCC is used to extract features. These features are adapted using minimum phone error (MPE) as a discriminative model. The acoustic model was built using the deep neural network (DNN) model. We present an n-gram language model (LM). The dataset of Qur’an verses is used for training and evaluating the proposed model, consisting of 10 hours of .wav recitations performed by 60 reciters. The Experimental results showed that the proposed DNN model achieved a significantly low character error rate (CER) of 4.09% and a word error rate (WER) of 8.46%.

N-gram models for Text Generation in Hindi Language

Native language plays a vital role for communication. Hindi is preferred by most Indians and it is the fifth most spoken language in the world. Hence, to make User Experience more effective while interacting with Software Applications, we aim to build a Model using Natural Language Processing which takes a specific word as an input and predicts the subsequent words for completing the sentence.It will act as a tab- complete function in Hindi language. This will also pave a way as a use case for building chatbots, personal emails, applications which are content based like cooking recipes, blogs, etc. in Hindi Language.

Adaptive Language Processing Unit for Malaysian Sign Language Synthesizer

Language Processing Unit (LPU) is a system built to process text-based data to comply with the rules of sign language grammar. This system was developed as an important part of the sign language synthesizer system. Sign language (SL) uses different grammatical rules from the spoken/verbal language, which only involves the important words that Hearing/Impaired Speech people can understand. Therefore, it needs word classification by LPU to determine grammatically processed sentences for the sign language synthesizer. However, the existing language processing unit in SL synthesizers suffers time lagging and complexity problems, resulting in high processing time. The two features, i.e., the computational time and success rate, become trade-offs which means the processing time becomes longer to achieve a higher success rate. This paper proposes an adaptive Language Processing Unit (LPU) that allows processing the words from spoken words to Malaysian SL grammatical rule that results in relatively fast processing time and a good success rate. It involves n-grams, NLP, and Hidden Markov Models (HMM)/Bayesian Networks as the classifier to process the text-based input. As a result, the proposed LPU system has successfully provided an efficient (fast) processing time and a good success rate compared to LPU with other edit distances (Mahalanobis, Levensthein, and Soundex). The system has been tested on 130 text-input sentences with several words ranging from 3 to 10 words. Results showed that the proposed LPU could achieve around 1.497ms processing time with an average success rate of 84.23% for a maximum of ten-word sentences.

Machine learning tool for exploring sentiment analysis on twitter data

In social media, micro-blogging is a common routine of most people around the world. Social media analytics gathering structure and unstructured Big-Data from various social sites and analyzing to make business decisions using Apache Hadoop and Apache Hive respectively. The objective of the research work was to develop big data technology used for gathering and handling large unstructured data from real time social media for sentiment analysis for identifying the brand and services. The methodology devised an algorithm based on sentimental analysis using customers review classification, which dealt with prepossessing the datasets, clustering of the data based on the specific domains, feature vector using n-gram models and tf-idf vectors extracts synonyms and classification sentiment analysis. The result shows that applied sentimental analysis with unsupervised clustering of data into specific domains and supervised machine learning techniques handle large amounts of twitter data in an efficient way. The developed tool 1.5 times faster than that of traditional database to Hadoop cluster and also the accuracy is nearly 80 %, which helps the user in computing, analyzing and interpreting interaction and associations between people, topics and ideas.

Solicitation of Knowledge Graph Enhanced Neural Network Objects Detection by Sentiment Analysis

In the machine learning technique, the knowledge graph is advancing swiftly; however, the basic models are not able to grasp all the affluence of the script that comes from the different personal web graphics, social media, ads, and diaries, etc., ignoring the semantic of the basic text identification. The knowledge graph provides a real way to extract structured knowledge from the texts and desire images of neural network, to expedite their semantics examination. In this study, we propose a new hybrid analytic approach for sentiment evaluation based on knowledge graphs, to identify the polarity of sentiment with positive and negative attitudes in short documents, particularly in 4 chirps. We used the tweets graphs, then the similarity of graph highlighted metrics and algorithm classification pertain sentimentality pre-dictions. This technique facilitates the explicability and clarifies the results in the knowledge graph. Also, we compare our differentiate the embeddings n-gram based on sentiment analysis and the result is indicated that our study can outperform classical n-gram models, with an F1-score of 89% and recall up to 90%.

Sentiment Analysis in Twitter Based on Knowledge Graph and Deep Learning Classification

The traditional way to address the problem of sentiment classification is based on machine learning techniques; however, these models are not able to grasp all the richness of the text that comes from different social media, personal web pages, blogs, etc., ignoring the semantic of the text. Knowledge graphs give a way to extract structured knowledge from images and texts in order to facilitate their semantic analysis. This work proposes a new hybrid approach for Sentiment Analysis based on Knowledge Graphs and Deep Learning techniques to identify the sentiment polarity (positive or negative) in short documents, such as posts on Twitter. In this proposal, tweets are represented as graphs; then, graph similarity metrics and a Deep Learning classification algorithm are applied to produce sentiment predictions. This approach facilitates the traceability and interpretability of the classification results, thanks to the integration of the Local Interpretable Model-agnostic Explanations (LIME) model at the end of the pipeline. LIME allows raising trust in predictive models, since the model is not a black box anymore. Uncovering the black box allows understanding and interpreting how the network could distinguish between sentiment polarities. Each phase of the proposed approach conformed by pre-processing, graph construction, dimensionality reduction, graph similarity, sentiment prediction, and interpretability steps is described. The proposal is compared with character n-gram embeddings-based Deep Learning models to perform Sentiment Analysis. Results show that the proposal is able to outperforms classical n-gram models, with a recall up to 89% and F1-score of 88%.

Review of Information Retrieval: Models, Performance Evaluation Techniques and Applications

Information Retrieval (IR) is a field that concerns the structure, memory, analysis, and access to pieces of information. It has a wide application in various areas like search engines, communication systems, information filtering, medical search, etc., and helps design efficient and secure applications. This area has been a surge of research from the last few years due to data mining's unparalleled success, deep learning in computer vision, blockchain technology, etc. Core models, performance evaluation techniques, IR system applications, and its role in blockchain technology have been proposed in this literature, calling the need for a broad survey to focus the research in this promising area. This paper fills the space by surveying the state of art approaches with deep learning models, query expansion techniques used, and the use of private information retrieval in blockchain technology. This survey paper includes different IR models like boolean model, vector space model, probabilistic model, language model, N-gram model, fuzzy model, Latent Semantic Indexing (LSI) Model, Bayesian network, Evolutionary algorithm-based models and Machine Learning based models. Applications of IR systems along with different datasets are also included to provide further research in this field.

Sparse User Check-in Venue Prediction By Exploring Latent Decision Contexts From Location-Based Social Networks

The proliferation of online Location-Based Social Networks (LBSN) has offered unprecedented opportunities for understanding fine-grained spatio-temporal behaviors of users and developing new location-aware applications. In this article, we focus on the problem of “Sparse User Check-in Venue Prediction,” where the goal is to predict the next venue LBSN users will visit by exploiting their sparse online check-in traces and the latent decision contexts. While efforts have been made to predict users’ check-in traces on a LBSN, several important challenges still exist. First, check-in traces contributed by LBSN users are often too sparse to provide sufficient evidence for a reliable prediction, especially when the prediction space is huge (e.g., hundreds of thousands of venues in large cities). Second, the user's decision context on which venue to visit next is often latent and has not been incorporated by current venue prediction models. Third, the dynamic and non-deterministic dependency between check-ins is either ignored or replaced by a simplified “consecutiveness” assumption in existing solutions, leading to sub-optimal prediction results. In this article, we develop a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Context-aware Sparse Check-in Venue Prediction (CSCVP)</i> scheme inspired by natural language processing techniques to address the above challenges. In particular, CSCVP predicts the venue category information and explores the similarity between users to address data sparsity challenge by significantly reducing the prediction space. It also leverages the Probabilistic Latent Semantic Analysis (PLSA) model to incorporate the user decision context into the prediction model. Finally, we develop a novel Temporal Adaptive Ngram (TA-Ngram) model in CSCVP to capture the dynamic and non-deterministic dependency between check-ins. We evaluate CSCVP using three real-world LBSN datasets. The results show that our scheme significantly improves accuracy (30.9 percent improvement) of the state-of-the-art user check-in venue prediction solutions.

Automatic detection of depression symptoms in twitter using multimodal analysis.

Depression is the most prevalent mental disorder that can lead to suicide. Due to the tendency of people to share their thoughts on social platforms, social data contain valuable information that can be used to identify user’s psychological states. In this paper, we provide an automated approach to collect and evaluate tweets based on self-reported statements and present a novel multimodal framework to predict depression symptoms from user profiles. We used n-gram language models, LIWC dictionaries, automatic image tagging, and bag-of-visual-words. We consider the correlation-based feature selection and nine different classifiers with standard evaluation metrics to assess the effectiveness of the method. Based on the analysis, the tweets and bio-text alone showed 91% and 83% accuracy in predicting depressive symptoms, respectively, which seems to be an acceptable result. We also believe performance improvements can be achieved by limiting the user domain or presence of clinical information.

Effects of Chord Inversion and Bass Patterns on Harmonic Expectancy in Musicians

This study tests the respective roles of pitch-class content and bass patterns within harmonic expectation using a mix of behavioral and computational experiments. In our first two experiments, participants heard a paradigmatic chord progression derived from music theory textbooks and were asked to rate how well different target endings completed that progression. The completion included the progression’s paradigmatic target, different inversions of that chord (i.e., different members of the harmony were heard in the lowest voice), and a “mismatched” target, a triad that shared its lowest pitch with the paradigmatic ending but altered other pitch-class content. Participants generally rated the paradigmatic target most highly, followed by other inversions of that chord, with lowest ratings generally elicited by the mismatched target. This suggests that listeners’ harmonic expectations are sensitive to both bass patterns and pitch-class content. However, these results did not hold in all cases. A final computational experiment was run to determine whether variations in behavioral responses could be explained by corpus statistics. To this end, n-gram chord-transition models and frequency measurements were compiled for each progression. Our findings suggest that listeners rate highly and have stronger expectations about chord progressions that occur frequently and behave consistently within tonal corpora.

Personalized API Recommendations

Application Programming Interfaces (APIs) play an important role in modern software development. Developers interact with APIs on a daily basis and thus need to learn and memorize those APIs suitable for implementing the required functions. This can be a burden even for experienced developers since there exists a mass of available APIs. API recommendation techniques focus on assisting developers in selecting suitable APIs. However, existing API recommendation techniques have not taken the developers personal characteristics into account. As a result, they cannot provide developers with personalized API recommendation services. Meanwhile, they lack the support for self-defined APIs in the recommendation. To this end, we aim to propose a personalized API recommendation method that considers developers’ differences. Our API recommendation method is based on statistical language. We propose a model structure that combines the N-gram model and the long short-term memory (LSTM) neural network and train predictive models using API invoking sequences extracted from GitHub code repositories. A general language model trained on all sorts of code data is first acquired, based on which two personalized language models that recommend personalized library APIs and self-defined APIs are trained using the code data of the developer who needs personalized services. We evaluate our personalized API recommendation method on real-world developers, and the experimental results show that our approach achieves better accuracy in recommending both library APIs and self-defined APIs compared with the state-of-the-art. The experimental results also confirm the effectiveness of our hybrid model structure and the choice of the LSTM’s size.

Research on attention and reasoning mechanism of autonomous English learning based on machine learning

The wave of the Internet era has swept over our lives and learning, and the Internet is gradually being widely used in all scenes of life and learning. The use of Internet technology can greatly improve efficiency compared to traditional methods. With the popularization of computers, internet technology has entered millions of households, and now more and more school courses are related to the Internet. However, the teaching evaluation of online learning is generally derived from the post-class feedback of teachers and students. In this paper, we analyze the video feeds of students during class and then qualitatively assess students' attention. In this paper, the English grammar error correction model is mainly based on deep learning, and the English grammar error correction problem is defined. The English grammar error correction evaluation criteria are given. Besides, the SEQ-based English grammar error correction model is proposed, and some improvements are made to the basic SEQ model, and then the algorithmic model is experimented and validated. Based on the above discussion, the feedback mechanism is proposed. Moreover, we also introduce the N-gram model and related implementation techniques, and experiments are conducted and experimental results are given with the corpus.

Classifier Performance Evaluation for Lightweight IDS Using Fog Computing in IoT Security

In this article, a Host-Based Intrusion Detection System (HIDS) using a Modified Vector Space Representation (MVSR) N-gram and Multilayer Perceptron (MLP) model for securing the Internet of Things (IoT), based on lightweight techniques and using Fog Computing devices, is proposed. The Australian Defence Force Academy Linux Dataset (ADFA-LD), which contains exploits and attacks on various applications, is employed for the analysis. The proposed method is divided into the feature extraction stage, the feature selection stage, and classification modeling. To maintain the lightweight criteria, the feature extraction stage considers a combination of 1-gram and 2-gram for the system call encoding. In addition, a Sparse Matrix is used to reduce the space by keeping only the weight of the features that appear in the trace, thus ignoring the zero weights. Subsequently, Linear Correlation Coefficient (LCC) is utilized to compensate for any missing N-gram in the test data. In the feature selection stage, the Mutual Information (MI) method and Principle Component Analysis (PCA) are utilized and then compared to reduce the number of input features. Following the feature selection stage, the modeling and performance evaluation of various Machine Learning classifiers are conducted using a Raspberry Pi IoT device. Further analysis of the effect of MLP parameters, such as the number of nodes, number of features, activation, solver, and regularization parameters, is also conducted. From the simulation, it can be seen that different parameters affect the accuracy and lightweight evaluation. By using a single hidden layer and four nodes, the proposed method with MI can achieve 96% accuracy, 97% recall, 96% F1-Measure, 5% False Positive Rate (FPR), highest curve of Receiver Operating Characteristic (ROC), and 96% Area Under the Curve (AUC). It also achieved low CPU time usage of 4.404 (ms) milliseconds and low energy consumption of 8.809 (mj) millijoules.