N-gram Model Research Articles

Automatic Semantic Annotation of Indonesian Language Phrase Using N-Gram Language Model

Building semantic data populations in unstructured data or text is challenging. In this type of data, several problems can be raised, some of which are difficult to analyze. Some groups of words or expressions cannot be defined according to their meaning and can be a source of ambiguity. It can have a different meaning depending on the context of its use. This work aims to automatically annotate Indonesian Language text, especially phrases, with the existing knowledge base. The result is text with semantic markup. Machines can automatically process this type of text because it describes its meaning. This work applies an n-gram language model to identify meaningful phrases and defines them as a unit so that every existing word or phrase is automatically semantically tagged. This work uses the DBpedia and schema.org knowledge base. The percentage of successfully labeled data in this job was 78% with 84.95% accuracy using DBpedia and 5.9% with 97.46% accuracy using schema .org. Some factors affect the accuracy score, including the availability of the required data with the data contained in the knowledge base, the system's ability in the POS tagging process, and many new terminology and local cultures that have not yet been contained in the knowledge bases, especially schema.org that is utilized as a standard for all search engines. This work will help the machine understand the semantics of text data. All pages obtained will be semantically tagged and, therefore, will be understood by machines. This ability will support the following processes.

Applying the Markov chain in natural language processing and three-pool model

This paper delves into the application of Markov chains in Natural Language Processing (NLP), and the Markov Chain Monte Carlo (MCMC) methodology relevant to the three-pool model. The former outlines the basic principles of Markov chains, highlighting their utility in predicting word sequences in language modelling and text generation, despite certain limitations. Also, the former describes mathematical frameworks like n-gram models that enhance prediction accuracy by considering multiple preceding words. It acknowledges challenges in NLP such as oversimplification and emotional depth, as well as computational issues in higher-order models. It concludes by discussing the integration of Markov chains with other models to mitigate these limitations, and their enduring relevance in computational linguistics. The later investigates the MCMC methodology, a seminal development in the field of statistical inference, which is especially useful when analysing complicated systems when traditional statistical techniques are inadequate. Moreover, this later explores the fundamental concepts of MCMC, clarifies how it is inherently related to Markov chains, presents the three-pool model that is commonly applied to models of physical, chemical, or ecological systems, and discusses how MCMC can be used to analyse these models.

A Knowledge-Driven Approach for Automatic Semantic Aspect Term Extraction Using the Semantic Power of Linked Open Data

Aspect-Based Sentiment Analysis (ABSA) is a crucial process for assessing customer feedback and gauging satisfaction with products or services. It typically consists of three stages: Aspect Term Extraction (ATE), Aspect Categorization Extraction (ACE), and Sentiment Analysis (SA). Various techniques have been proposed for ATE, including unsupervised, supervised, and hybrid methods. However, many studies face challenges in detecting aspect terms due to reliance on training data, which may not cover all multiple aspect terms and relate semantic aspect terms effectively. This study presents a knowledge-driven approach to automatic semantic aspect term extraction from customer feedback using Linked Open Data (LOD) to enrich aspect extraction outcomes in the training dataset. Additionally, it utilizes the N-gram model to capture complex text patterns and relationships, facilitating accurate classification and analysis of multiple-word terms for each aspect. To assess the effectiveness of the proposed model, experiments were conducted on three benchmark datasets: SemEval 2014, 2015, and 2016. Comparative evaluations with contemporary unsupervised, supervised, and hybrid methods on these datasets yielded F-measures of 0.80, 0.76, and 0.77, respectively.

MSR82 Implementation and Validation of the Use of N-Gram Models for Classifying Social Determinants of Health Status from Real World Unstructured Clinical Notes

MSR82 Implementation and Validation of the Use of N-Gram Models for Classifying Social Determinants of Health Status from Real World Unstructured Clinical Notes

MSR62 Clinical Validation of an N-Gram Model for Detecting Economic Insecurity in Patient Populations Using Unstructured Clinical Notes

MSR62 Clinical Validation of an N-Gram Model for Detecting Economic Insecurity in Patient Populations Using Unstructured Clinical Notes

Hybrid Natural Language Processing Model for Sentiment Analysis during Natural Crisis

This paper introduces a novel natural language processing (NLP) model as an original approach to sentiment analysis, with a focus on understanding emotional responses during major disasters or conflicts. The model was created specifically for Croatian and is based on unigrams, but it can be used with any language that supports the n-gram model and expanded to multiple word sequences. The presented model generates a sentiment score aligned with discrete and dimensional emotion models, reliability metrics, and individual word scores using affective datasets Extended ANEW and NRC WordEmotion Association Lexicon. The sentiment analysis model incorporates different methodologies, including lexicon-based, machine learning, and hybrid approaches. The process of preprocessing includes translation, lemmatization, and data refinement, utilized automated translation services as well as the CLARIN Knowledge Centre for South Slavic languages (CLASSLA) library, with a particular emphasis on diacritical mark correction and tokenization. The presented model was experimentally evaluated on three simultaneous major natural crises that recently affected Croatia. The study’s findings reveal a significant shift in emotional dimensions during the COVID-19 pandemic, particularly a decrease in valence, arousal, and dominance, which corresponded with the two-month recovery period. Furthermore, the 2020 Croatian earthquakes elicited a wide range of negative discrete emotions, including anger, fear, and sadness, with the recuperation period much longer than in the case of COVID-19. This study represents an advancement in sentiment analysis, particularly in linguistically specific contexts, and provides insights into the emotional landscape shaped by major societal events.

Prediction of Next Word in Balochi Language Using N-gram Model

Balochi Language is among the oldest languages, spoken by approximately 10 million people worldwide. The Balochi language has been spoken for a very long period. In comparison to other languages like English, Urdu, French etc. it has a research gap in Natural language processing (NLP). The next word prediction system is one of the techniques of NLP for suggesting standardization and corpus collection. This research aims to provide a next-word prediction system and a corpus with no ambiguity for the Balochi language. N-gram model for the next word prediction has been utilized, i.e. Unigram, Bigram, Trigram, Quad-gram, and so on. A trained model has been embedded in an application after being evaluated extrinsically and intrinsically. It plays a crucial role in typing through a keyboard and helps users to type faster. Additionally, it helps native users to have fewer typing errors in less time. The results of the research show that Five-gram model has the highest performance of 93% while Quad-gram model has 80% and Trigram model has 76% respectively.

Features level sentiment mining in enterprise systems from informal text corpus using machine learning techniques

ABSTRACT This study explores feature-level sentiment analysis of Hindi blog reviews in enterprise systems, a significant area in the Indian context yet understudied. By applying machine learning techniques like SVM across unigram, bigram, trigram, and n-gram models, and combining Lexicon-based methods with machine learning algorithms, we aim to enhance sentiment classification for better customer relationship management and product development. Contrasting with document-level approaches, our method focusing on bigrams achieves a test accuracy of 75%, offering a scalable model for enterprises to extract detailed customer insights from informal text, thereby aiding informed decision-making in a multicultural environment.

Detecting unknown vulnerabilities in smart contracts using opcode sequences

Unknown vulnerabilities, also known as zero-day vulnerabilities, are vulnerabilities in software, systems, or networks that have not yet been publicly disclosed or fixed. If these vulnerabilities are ever discovered by hackers, intentionally or unintentionally, they pose a major threat to network security. This is particularly true in the blockchain field, as smart contracts hold a lot of money, and if they are discovered and exploited by hackers, the financial losses to users will be even greater. However, the current research on smart contract vulnerabilities mainly focuses on known vulnerabilities, and the research on unknown vulnerabilities has been limited. Based on this, we introduce a machine learning-based method for detecting unknown vulnerabilities in smart contracts. First, the method obtains the opcode sequences executed by smart contract transactions in the EVM by instrumenting Geth and replaying the Ethereum transactions. Next, we employ an n-gram model and a vector weight penalty mechanism to extract the opcode sequence features. We then use machine learning algorithms to detect unknown vulnerabilities based on the similarity principle. Finally, we test the effectiveness of our method with four machine learning models: the K-Nearest Neighbor algorithm (KNN), Support Vector Machine (SVM), Logistic Regression (LR), and Decision Tree (DT). The SVM model performs best at detecting unknown vulnerabilities, with an accuracy of 96%, a precision of 91%, a recall of 100%, and an F1-score of 95%. We also discuss the benefits of the method: timely detection of attacks due to unknown vulnerabilities, thus reducing user losses.

N-gram Language Model for Chinese Function-word-centered Patterns

N-gram language modelling, a proven and effective method in NLP, is widely used to calculate the probability of a sentence in natural language. Language pattern is a linguistic level between word/character and sentence, which exists in pattern grammar. In this research, the approach of language model and language pattern are combined for the first time, and language patterns are studied by use of the N-gram model. Chinese function-word-centered patterns are extracted from the LCMC corpus, and aligned into pattern chains. The language model is trained from these chains to investigate the properties and distribution of Chinese function words, the interaction of content words and function words, and the interaction between patterns. The results indicate that there are approximately 10,000 function-word-centered patterns in the texts, which are distributed exponentially. This research summarizes the most common function-word-centered patterns and content-word-centered patterns, and discusses the interactions of patterns based on corpus data. The bigram language model of these patterns reflects the restrictions of function words. In addition, the research adopts an innovative method to visualize the interactions between patterns. This research fills the research gap between word/character and sentence and reveals basic Chinese pattern categories and the interactions between patterns, which makes a significant contribution to Chinese linguistic research, and improves the efficiency of NLP.

Melodic Differences Between Styles: Modeling Music With Step Inertia

A well-known phenomenon in melodic structure is “step inertia”: the tendency for a step to be followed by another step in the same direction. There is strong evidence of step inertia in three corpora of Western common-practice melodies: European folk songs, classical instrumental themes, and English hymn tunes. Surprisingly, modern Western popular music does not reflect step inertia. In Billboard’s Hot 100, and Rolling Stone magazine's list of “greatest songs,” inertial (same-direction) steps are less likely than non-inertial ones. To further explore the role of step inertia in different corpora, we created a generative model that assigns probabilities to melodies, considering just four factors: range, pitch proximity, scale-degree frequency within a key, and step inertia. We optimized the weights of these factors for the Essen Folksong Collection and the Billboard corpus, and compared them with n-gram models. The optimal (normalized) weight of the inertia factor is large and positive for the Essen collection (.51) and small for the Billboard corpus (.02). This is further evidence that step inertia plays a much smaller role in popular melodies than common-practice ones, and that non-inertial steps are slightly favored.

Optimization of Logical Structure and Innovative Practice of Law-Based Evidence Talent Cultivation in the Context of Education-Industry Integration

Abstract The construction of “new liberal arts” and the advent of the era of artificial intelligence are forcing the traditional training mode of legal talents to change in the direction of interdisciplinary, high-quality composite talents who know how to govern society, among which, the legal evidence-based talents are the talents training direction adapted to the future needs and have a noble sense of social responsibility. At present, the cultivation of evidence-based talents is still in the exploratory stage, and in order to use the specific cultivation method, we need to cooperate with high-quality enterprises and practical departments. Practical exploration reveals that, on the one hand, utilizing the technical resources of enterprises can effectively enhance students’ ability to use evidence-based research methods. On the other hand, in cooperation with practical departments, conducting research on the rule of law around real issues and evaluating the implementation effect of policies can well exercise students’ practical ability and deepen their patriotic sentiment of helping people in the world. In addition, this paper also carries out the vectorized representation of legal text based on the N-Gram model, calculates the similarity of legal cases by combining with Word2Vec, and introduces the graph representation learning to establish a similar case law recommendation model, which helps students to find out the similar cases in the massive cases. It is conducive to students’ cultivation of systematic thinking in evidence-based jurisprudence and enhancement of their ability to discriminate similar cases. Through the practice of the course, the initial feasible experience of “five-stage parallel progress” and “three integration of science and reality” on the cultivation of evidence-based law talents has been formed, and the incentives and assessment mechanisms for the cultivation of industry-academia integration need to be further strengthened in the future.

Practices and Challenges of Artificial Intelligence-Assisted Teaching in Vocational Undergraduate Public English Classrooms

Abstract This paper first introduces the architecture of the natural language processing system and then analyzes how natural language processing decomposes the probability distribution function by multi-factor form and Bayesian formula to improve the efficiency of function description. Then, NLM is applied to enhance the statistical coefficients of the N-gram model, which solves the problems of data sparsity and dimensionality disaster. By using natural language understanding in English teaching practice and experimenting with the teaching effect, it was found that the overall English achievement of the experimental group increased by 5.4 points, the average Z-score reached more than 3 points, and 96.6% of the students were interested in the teaching method. The new teaching method is demonstrated to have a significant impact on the improvement of English scores.

Contemporary Approaches in Evolving Language Models

This article provides a comprehensive survey of contemporary language modeling approaches within the realm of natural language processing (NLP) tasks. This paper conducts an analytical exploration of diverse methodologies employed in the creation of language models. This exploration encompasses the architecture, training processes, and optimization strategies inherent in these models. The detailed discussion covers various models ranging from traditional n-gram and hidden Markov models to state-of-the-art neural network approaches such as BERT, GPT, LLAMA, and Bard. This article delves into different modifications and enhancements applied to both standard and neural network architectures for constructing language models. Special attention is given to addressing challenges specific to agglutinative languages within the context of developing language models for various NLP tasks, particularly for Arabic and Turkish. The research highlights that contemporary transformer-based methods demonstrate results comparable to those achieved by traditional methods employing Hidden Markov Models. These transformer-based approaches boast simpler configurations and exhibit faster performance during both training and analysis. An integral component of the article is the examination of popular and actively evolving libraries and tools essential for constructing language models. Notable tools such as NLTK, TensorFlow, PyTorch, and Gensim are reviewed, with a comparative analysis considering their simplicity and accessibility for implementing diverse language models. The aim is to provide readers with insights into the landscape of contemporary language modeling methodologies and the tools available for their implementation.

RESUME SUMMARIZER AND JOB DESCRIPTION MATCHER USING NATURAL LANGUAGE PROCESSING AND SPACY

In the HR recruitment process, one pivotal stage is "Summarizing & Screening." At this juncture, recruiters grapple with the laborious task of manually sifting through a myriad of resumes to shortlist suitable candidates. This project aims to streamline and automate this process using two innovative tools. Firstly, the "Resume Summarizer" leverages advanced natural language processing techniques to swiftly extract pertinent details from candidates' resumes, offering a concise summary that underscores their skills, experience, and qualifications. This summary can be tailored to match specific job requirements, facilitating recruiters in the comparative evaluation of multiple candidates. Secondly, the "Job Description Matcher" assists recruiters in expeditiously and accurately aligning job descriptions with candidate resumes. By employing natural language processing, this tool identifies keywords and phrases that align with the job opening's prerequisites, generating a compatibility score to rank resumes according to their alignment with the job description. This innovation greatly expedites candidate selection and enhances recruiters' efficiency. Importantly, this project employs a unique approach, incorporating regular expressions to extract essential information from resumes, setting it apart from existing NLP-based models and systems. Keywords: Natural Language Processing, Text Analysis, Data Extraction, Keyword matching, Regular Expressions, Optical Character Recognition (OCR), N-Gram Model, Name Entity Recognition (NER), Unsupervised NER.

Languages with more speakers tend to be harder to (machine-)learn

Computational language models (LMs), most notably exemplified by the widespread success of OpenAI's ChatGPT chatbot, show impressive performance on a wide range of linguistic tasks, thus providing cognitive science and linguistics with a computational working model to empirically study different aspects of human language. Here, we use LMs to test the hypothesis that languages with more speakers tend to be easier to learn. In two experiments, we train several LMs—ranging from very simple n-gram models to state-of-the-art deep neural networks—on written cross-linguistic corpus data covering 1293 different languages and statistically estimate learning difficulty. Using a variety of quantitative methods and machine learning techniques to account for phylogenetic relatedness and geographical proximity of languages, we show that there is robust evidence for a relationship between learning difficulty and speaker population size. However, contrary to expectations derived from previous research, our results suggest that languages with more speakers tend to be harder to learn.

DETECTING DECEPTIVE REVIEWS: AN INTEGRATED MACHINE LEARNING APPROACH

In recent years, online reviews have become a crucial factor in promoting products and services. However, the rise of fake reviews has posed a significant challenge. Businesses, marketers, and advertisers often resort to embedding fake reviews to attract customers or undermine their competitors. Deceptive reviews have become a common practice, as they serve as a means of promoting one''''s own business or tarnishing the reputation of rivals. Consequently, the identification of deceptive reviews has emerged as a critical and ongoing research area. This research paper presents a machine learning model approach to detect deceptive reviews. The study focuses on experiments conducted using a deceptive opinion spam corpus dataset, specifically targeting restaurant reviews. An n-gram model combined with max features is developed to identify deceptive content, with a particular emphasis on fake reviews. Additionally, a benchmark study is conducted to explore the performance of two different feature extraction techniques and their application in five machine learning classification techniques. The experimental findings demonstrate that the passive aggressive classifier outperforms other algorithms, achieving the highest accuracy not only in text classification but also in identifying fake reviews. Moreover, the research delves into the identification of deceptive reviews and explores diverse feature extraction and machine learning techniques to improve the model''''s accuracy.

Statistical language model-based analysis of the English-Chinese corpus and political discourse

Abstract Politics and political discourse are closely related to people’s daily life, and this study aims to propose a new approach to political discourse analysis by combining English and Chinese corpora. By exploring the composition of formal language and the grammar generation process, this paper proposes an improved N-gram algorithm to address the shortcomings of the N-gram model in dealing with low-frequency words with low accuracy and uses the strategy of introducing alternative words to alleviate the problem of sparse data. Then, a critical metaphor analysis of political discourse in the English-Chinese corpus is conducted based on the improved statistical language model, and the convergence of political discourse is studied in terms of space and time. By analyzing the political discourse of American presidents, the spatial centrality factors of “we” and “our nation” were accurately extracted, and their correlations were 0.83, 0.73, 0.68, 0.51, 0.76, and 0.41 in order. The correlations of the unqualified facsimile noun phrases in the temporal convergence of political discourse reached 0.28, 0.25, 0.72, 0.68, and 0.54, respectively, and the accuracy of the improved N-gram model improved by about 28.1% compared with the traditional method, making using statistical linguistic models for political discourse analysis feasible and applicable.

Keyword Extraction-based Library Intelligence Services: Challenges, Adaptations and Reinvention

Abstract This paper proposes a domain knowledge base construction platform for thematic intelligence services of library intelligence institutions. The platform proposes a keyword extraction algorithm applicable to the literature resources of library intelligence services based on a statistical model and N-Gram model combined with feature word information. The automatic summary algorithm based on statistics combines the information of sentence content and location to comprehensively evaluate the weights of candidate summary sentences and achieve automatic summary extraction. Based on the HDP topic model, a literature recommendation model oriented to the resource retrieval context is proposed, which can convert the literature feature similarity calculation to the topic similarity calculation with implied literature feature semantics from the perspective of semantic analysis and improve the accuracy of recommended literature. The technical feasibility of this system platform is verified through relevant data experiments. Regarding the accuracy of the literature recommendation, the platform of this paper and the traditional method are 86.36% and 75.16%, respectively, and the recall rate is 67.29% and 57.25%, respectively. The platform scored 3.85 on the comprehensive evaluation of library intelligence service, close to the excellent level. Therefore, the platform can realize various functions, from resource collection and organization to resource processing and utilization, with certain practical significance.

A Malicious Mining Code Detection Method Based on Multi-Features Fusion

With the continuous increase in the economic value of new digital currencies represented by Bitcoin, more and more cybercriminals use malicious code to occupy victims system resources and network resources for mining without the victims permission, thereby obtaining cryptocurrency. This type of malicious code named malicious mining code has brought considerable influence and harm to society, enterprises and users. The mining code always conceals the fact that it consumes computer resources in a way that is difficult for ordinary people to discover. This paper proposes a malicious mining code detection method based on feature fusion and machine learning. First, we analyze from static analysis methods and statistical analysis methods to extract multi-dimensional features. Then for multi-dimensional text features, feature vectors are extracted through the n-gram model and TF-IDF, and best feature vectors are selected through the classifier and we fuse these best feature vectors with other statistic features to train our detection model. Finally, automatic detection is performed based on the machine learning framework. The experimental results show that the recognition accuracy of our method can reach 98.0%, its f1 score reach 0.969, and the ROCs AUC reach 0.973.