Probabilistic Latent Semantic Analysis Research Articles

Аддитивная регуляризация при тематическом моделировании текстов сообществ онлайновых социальных сетей

The task of modeling communities (groups) of users in social media is relevant in the framework of information support for decision-making at different levels of government. For automated extraction of the meaning of textual and related information, topic modeling methods are used. This article presents the experience of improving the results of social networks communities topic modeling using the Additive Regularization for Topic Modeling (ARTM). The improvement of the results is achieved through the use of basic regularizers available in the open-source software BigARTM. Topic models obtained using regularized ARTM are compared with topic models obtained by Latent Dirichlet Allocation and Probabilistic Latent Semantic Analysis. The experiments were carried out on a dataset, containing preprocessed texts of posts from communities of an online social network. In particular, the quality of topic models in terms of coherence, the purity of topics, and the sparsity of the distribution matrices are compared. Disadvantages of coherence metrics for assessing the quality of topic models obtained using the ARTM method are discussed. Additional metrics are proposed that can be used for assessing the quality of topic models. Conclusions are drawn about the suitability of the ARTM approach for modeling communities of online social networks. The results of this work can be applied in the development of information and analytical systems for supporting the management of regional development.

TOPIC MODELING BY USING LDA

Topic modelling plays a significant Information retrieval showed good performance on a wide variety of tasks over the years. It has wide research in machine learning and text mining. The combination of the probability mixture model and the term model presents the topic model. They are the language modelling framework of information retrieval. The present topic modelling methods are probabilistic latent semantic analysis and latent dirichlet allocation. Two main faults of topic modelling are First, common or popular words are different topics, often causing ambivalence to understand the topic. Second, single words have to be presented correctly. Actual problems of topic modelling are Efficiency and scaling are discussed and compared in different types of Topic modelling. The review of article is about how topic modelling for information retrieval is done using LDA(Latent Dirichlet Allocation).

Effect of Preprocessing and No of Topics on Automated Topic Classification Performance

The emergence of the Internet has caused an increasing generation of data. A high amount of the data is of textual form, which is highly unstructured. Almost every field i.e, business, engineering, medicine, and science can benefit from the textual data when knowledge is extracted. The knowledge extraction requires the extraction and recording of metadata on the unstructured text documents that constitute the textual data. This phenomenon is regarded as topic modeling. The resulting topics can ease searching, statistical characterization, and classification. Some well-known algorithms for topic modeling include Latent Dirichlet Allocation (LDA), Nonnegative Matrix Factorization (NMF), and Probabilistic Latent Semantic Analysis (PLSA). Different parameters can affect the performance of topic modeling. An interesting parameter could be the time required to perform topic modeling. The fact that time is affected by many factors applicable to topic modeling as well; however, measuring the time concerning some constraints can be beneficial to provide insight. In this paper, we alter some preprocessing steps and topics to study their impact on the time taken by the LDA and NMF topic models. In preprocessing, we limit our study by altering only the sampling and feature subset selection whereas in the second step we have changed the number of topics. The results show a significant improvement in time.

METHODS OF CLASSIFICATION OF MACHINE LEARNING FOR CONSTRUCTION OF MATHEMATICAL MODELS ON MULTIMODAL DATA

This article is dedicated to topic modeling as an unsupervised machine learning technique. It is analyzed how it seems possible to determine the topics of documents in order to categorize them further with the help of topic modeling methods. Such methods as latent semantic analysis, probabilistic latent semantic analysis and latent Dirichlet allocation are considered. An approach that allows the construction of effective topic models of text document collections in Ukrainian and other synthetic languages based on peculiarities of this linguistic language type is proposed, and its main stages are described. The proposed approach consists of a custom input data preprocessing pipeline, which covers file loading, text extraction, removal of improper symbols, tokenization, removal of stop-words, stemming of each token and a newly introduced model pruning stage, which makes any of the modern topic modeling methods applicable for synthetic language topic modeling. The approach was implemented in Python programming language and used to obtain the topic model of the collection of Ukrainian-language scientific publications on civic identity and related topics. An expert in political psychology, who studies the phenomenon of civic identity, was involved in the research for the topic model quality evaluation. As a result of expert evaluation of the topics singled out during the modeling, it was proposed to clarify the formulation of cluster names based on the semantics of the sets of words that form them. In general, according to the expert, the topics singled out represent the concept of the civic identity of an individual and will allow researchers to simplify the work with literature sources on this issue when used to categorize documents. This demonstrates the efficiency of the proposed approach.

COMPARATIVE ANALYSIS OF THEMATIC MODELING METHODS FOR ANALYSIS OF REVIEWS IN THE ONLINE STORE OF DIGITAL GOODS

Nowadays, people often use online services for their daily tasks. The Internet has increased the demand for applications and services to provide a better customer experience. However, nowadays the Internet is full of information that can make it difficult to understand customer needs and confuse users when searching for the information they need. Therefore, there is a need to use effective methods and tools that can help in identifying and analyzing information from a large number of sources stored as online text. For such tasks, it is convenient to use natural language processing – an industry that combines the capabilities of computational linguistics, computer science and artificial intelligence to allow computer to understand and analyze meaning of human speech. One of the fundamental tasks of natural language processing is the definition of keywords. Identified keywords are used to determine the needs of users of the product when it comes to analyzing product reviews, and quickly find information about the product by the average user. Topic modeling methods are often used to determine keywords in the text This study provides a comparative analysis of topic modeling methods for use in text documents taken from reviews of digital products in the online store. Topic modeling is an unsupervised machine learning technique that allows you to analyse collection of documents and divide them into different topics. Three of the most popular topic modeling methods presented in this paper for document research are latent semantic analysis LSA, probabilistic latent semantic analysis PLSA, and latent Dirichlet allocation LDA. Comparative analysis is performed using numerical metrics such as coherence, perplexity and “human eye” evaluation metrics using word cloud visualization of results for different parameters of these methods. In addition, a comparison of performance methods was performed.

A Behavior Analysis of the Impact of Semantic Relationships on Topic Discovery

Information Technologies have generated large amounts of documents available for analysis and use. Information systems can provide the user with the necessary data for a specific purpose without human intervention, saving time in providing the response expected by the user. Some traditional models of topic discovery provide essential information in the literature, but it is still necessary to incorporate the knowledge that a person can use when reading a document. In this work, an analysis of the behavior of the techniques of Latent Dirichlet Analysis, Latent Semantic Analysis, and Probabilistic Latent Semantic Analysis is carried out, incorporating the semantic relationships of the type hypernym, hyponym, synonymy, holonymy, and meronymy extracted from the external source of knowledge WordNet, in order to improve the results obtained by applying the three mentioned techniques in a set of documents without adding external knowledge. The experimental results improved when incorporating semantic relationships such as hypernyms and synonyms compared to the initial results, but the best result was when using a disambiguation algorithm Lesk and subsequently applying Latent Dirichlet Analysis.

Analyzing Machine Learning Enabled Fake News Detection Techniques for Diversified Datasets

Fake news, or fabric which appeared to be untrue with point of deceiving the open, has developed in ubiquity in current a long time. Spreading this kind of data undermines societal cohesiveness and well by cultivating political division and doubt in government. Since of the sheer volume of news being disseminated through social media, human confirmation has ended up incomprehensible, driving to the improvement and arrangement of robotized strategies for the recognizable proof of wrong news. Fake news publishers use a variety of stylistic techniques to boost the popularity of their works, one of which is to arouse the readers’ emotions. Due to this, text analytics’ sentiment analysis, which determines the polarity and intensity of feelings conveyed in a text, is now being utilized in false news detection methods, as either the system’s foundation or as a supplementary component. This assessment analyzes the full explanation of false news identification. The study also emphasizes characteristics, features, taxonomy, different sorts of data in the news, categories of false news, and detection approaches for spotting fake news. This research recognized fake news using the probabilistic latent semantic analysis approach. In particular, the research describes the fundamental theory of the related work to provide a deep comparative analysis of various literature works that has contributed to this topic. Besides this, a comparison of different machine learning and deep learning techniques is done to assess the performance for fake news detection. For this purpose, three datasets have been used.

Research on high-performance English translation based on topic model

Retelling extraction is an important branch of Natural Language Processing (NLP), and high-quality retelling resources are very helpful to improve the performance of machine translation. However, traditional methods based on the bilingual parallel corpus often ignore the document background in the process of retelling acquisition and application. In order to solve this problem, we introduce topic model information into the translation mode and propose a topic-based statistical machine translation method to improve the translation performance. In this method, Probabilistic Latent Semantic Analysis (PLSA) is used to obtains the co-occurrence relationship between words and documents by the hybrid matrix decomposition. Then we design a decoder to simplify the decoding process. Experiments show that the proposed method can effectively improve the accuracy of translation.

Personalized Recommendation Algorithm for University Civics Courses with Multiple User Interests

In response to the problems of unity, lack of relevance, lack of synergy, and inability to form a personalized collaborative education mechanism in the current curriculum setting of university thinking and politics education, a personalized recommendation system for university thinking and politics courses based on the multiple interests of users was developed. The system of interests is divided into two parts: first is initial interest guidance, in which the N meta-model is used to learn the context of known course processes; second is user interest extraction; at the end of creating the recommendation process, facing the diversity of user interests, probabilistic latent semantic analysis trains the interest-service-flow distribution of students to recommend the civics course that matches the current interest for students. A good recommendation algorithm can simulate learners’ enthusiasm and give full play to different learners’ learning personalities. The simulation experiments show that the system is stable in operation, complete in function, and has strong practicality and robustness, which is of positive significance in creating a win-win, diverse, and innovative atmosphere for students’ and teachers’ thinking education in colleges and universities.

Improving sentiment analysis with learning concepts from concept, patterns lexicons and negations

The way of expressing sentiment (−ve/+ve) in the form of textual information depends on the way of thinking of human beings. Identifying aspect extraction and sentiment polarity from written texts is a crucial task. Mainly, a multi-level learning approach for aspect extraction from statistical methods, pattern-based methods, and rule-based methods. This work proposes the application of two probabilistic graphical Latent Dirichlet Allocation (LDA) and Probabilistic Latent Semantic Analysis (PLSA) algorithms to generate latent topic terms as possible aspects. Then frequency-based and Concept lexicons are used to retrieve unigram to multi-word phrases with associated opinion words. Polarity shift is a significant issue that reverses the polarity of the aspects that affect the sentiment classification of the system. Therefore, to improve the performance of the machine learning classification algorithm in ABSA a hybrid approach comprising rule-based methods and a graph-theoretic model is applied to deal with the explicit and implicit polarity shift. The performance of the proposed method is measured using Naive Bayes, a machine learning classification algorithm on two datasets, SemEval 2014 Restaurant and SemEval 2014 Laptop dataset. Experimental result shows that the method for aspect extraction outperforms baseline methods by 86.32% and 82.64% for Restaurant, Laptop dataset, respectively. Similarly, for aspect-based sentiment classification, the accuracy and F1 measure on Restaurant domain 84.73%, 81.28% and 82.06% and 80.71% on the laptop domain.

Research on the Artistic Conception of Multimedia-Assisted Ancient Poetry Based on AI Technology

In order to improve the visual display effect of the artistic conception of ancient poems, this paper combines AI technology to construct a multimedia-assisted artistic conception creation system of ancient poetry and combines probabilistic latent semantic analysis to analyze the semantic artistic conception of ancient poetry. Moreover, with the support of multimedia simulation technology, this paper improves the multimedia teaching mode to improve the effect of multimedia-assisted ancient poetry teaching. In addition, this paper constructs a multimedia-assisted artistic conception creation system of ancient poetry based on AI technology and designs experiments to conduct system performance research. The experimental research results show that the multimedia-assisted artistic conception creation system of ancient poetry proposed in this paper has a good teaching effect and has an important role in promoting the study of the artistic conception of ancient poetry.

Investigating the Efficient Use of Word Embedding with Neural-Topic Models for Interpretable Topics from Short Texts.

With the rapid proliferation of social networking sites (SNS), automatic topic extraction from various text messages posted on SNS are becoming an important source of information for understanding current social trends or needs. Latent Dirichlet Allocation (LDA), a probabilistic generative model, is one of the popular topic models in the area of Natural Language Processing (NLP) and has been widely used in information retrieval, topic extraction, and document analysis. Unlike long texts from formal documents, messages on SNS are generally short. Traditional topic models such as LDA or pLSA (probabilistic latent semantic analysis) suffer performance degradation for short-text analysis due to a lack of word co-occurrence information in each short text. To cope with this problem, various techniques are evolving for interpretable topic modeling for short texts, pretrained word embedding with an external corpus combined with topic models is one of them. Due to recent developments of deep neural networks (DNN) and deep generative models, neural-topic models (NTM) are emerging to achieve flexibility and high performance in topic modeling. However, there are very few research works on neural-topic models with pretrained word embedding for generating high-quality topics from short texts. In this work, in addition to pretrained word embedding, a fine-tuning stage with an original corpus is proposed for training neural-topic models in order to generate semantically coherent, corpus-specific topics. An extensive study with eight neural-topic models has been completed to check the effectiveness of additional fine-tuning and pretrained word embedding in generating interpretable topics by simulation experiments with several benchmark datasets. The extracted topics are evaluated by different metrics of topic coherence and topic diversity. We have also studied the performance of the models in classification and clustering tasks. Our study concludes that though auxiliary word embedding with a large external corpus improves the topic coherency of short texts, an additional fine-tuning stage is needed for generating more corpus-specific topics from short-text data.

Email thread sentiment sequence identification using PLSA clustering algorithm

Email messaging is the most common way of providing effective communication between internauts. Consequently, the total sent and received emails count will be increased. But, the internaut can't remember all such emails. Even though email thread identification approaches give satisfactory benefits to the internauts, but they may fail to alert them for a cause to identify the sentiments behind an email thread. To address, this issue Probabilistic Latent Semantic Analysis clustering algorithm has been used in this paper to identify the email sentiment thread sequence. The sentiment and the thread sequence within the emails have been discovered as clustering sentiment polarity and temporal categories with the help of PLSA clusters. At the initial stage, we used three feature extraction methods, latent semantic analysis (LDA), bag of words (BoW), TF-IDF and SentiWordNet (SWN) lexicon for generating sentiment features of email. Next, Probabilistic Latent Semantic Analysis algorithm is used to form email clusters based on sentiment features. Thus, it helps to identify thread sentiment and sequence of sentiment threads. Email threads give a mechanism by which any user will be able to find out the sequence in the thread on the basis of sentiment analysis of email related to a specific set of communication during a specific time period. Various parameters evaluation measures have been considered in this work to evaluate the proposed model such as accuracy, precision, recall and F-measure, and the proposed algorithm is compared with other standard algorithms. Furthermore, a statistical test has also been performed.

Transfer Learning Based on Probabilistic Latent Semantic Analysis for Analyzing Purchase Behavior Considering Customers' Membership Stages

Transfer Learning Based on Probabilistic Latent Semantic Analysis for Analyzing Purchase Behavior Considering Customers' Membership Stages

Role of Genetic Algorithm in Optimization of Hindi Word Sense Disambiguation

The Word Sense Disambiguation system (WSD) is widely used in many fields, including business, research, education, and government organizations. The availability of natural language data on the internet has grown in tandem with the rapid advancement of technology and the widespread use of the internet. Disambiguation is best tool for avoiding the problem of misperception. The most important feature of this approach is that words with multiple meanings of nouns can be easily disambiguated according to the sense of the sentences. As a result of incorporating all of these factors, This work will concentrate on the accuracy of adjectives and verbs. Many natural languages, including English, German, Arabic, Assamese, and others, have different methodologies proposed, however, work on Word Sense Disambiguation in Hindi is limited. For Hindi Word Sense Disambiguation, the proposed algorithm employs a genetic algorithm. The dynamic configuration window function, including the ambiguous terms left and right, is used. The central tenet of this method is that the target term and its surroundings share a common topic. With approximately 65.17%, 72%, and 74.1 %, respectively, the proposed work has compared to existing approaches such as graph-based approaches, association rules approaches, and probabilistic latent semantic analysis approaches. The proposed method has an accuracy of 80%, which indicating that it improves the existing Work by 8% in terms of accuracy and other parameters. Results show that the proposed work generates more accurate results for Hindi word sense disambiguation.

Deep unsupervised methods towards behavior analysis in ubiquitous sensor data

Behavioral analysis (BA) on ubiquitous sensor data is the task of finding the latent distribution of features for modeling user-specific characteristics. These characteristics, in turn, can be used for a number of tasks including resource management, power efficiency, and smart home applications. In recent years, the employment of topic models for BA has been found to successfully extract the dynamics of the sensed data. Topic modeling is popularly performed on text data for mining inherent topics. The task of finding the latent topics in textual data is done in an unsupervised manner. In this work we propose a novel clustering technique for BA which can find hidden routines in ubiquitous data and also captures the pattern in the routines. Our approach efficiently works on high dimensional data for BA without performing any computationally expensive reduction operations. We evaluate three different techniques namely Latent Dirichlet Allocation (LDA), the Non-negative Matrix Factorization (NMF), and the Probabilistic Latent Semantic Analysis (PLSA) for comparative study. We have analyzed the efficiency of the methods by using performance indices like perplexity and silhouette on three real-world ubiquitous sensor datasets namely, the Intel Lab, Kyoto, and MERL. Through rigorous experiments, we achieve silhouette scores of 0.7049 over the Intel Lab dataset, 0.6547 over the Kyoto dataset, and 0.8312 over the MERL dataset for clustering. In these cases, however, it is di cult to validate the results obtained as the datasets do not contain any ground truth information. Towards that, we investigate a self-supervised method that will be capable of capturing the inherent ground truths that are available in the dataset. We design a self-supervised technique which we apply on datasets containing ground truth and also without. We see that our performance on data without ground truth differs from that with ground truth by approximately 8% (F-score) hence showing the efficacy of self-supervised techniques towards capturing ground truth information.

Bert-Based Latent Semantic Analysis (Bert-LSA): A Case Study on Geospatial Data Technology and Application Trend Analysis

Geospatial data is an indispensable data resource for research and applications in many fields. The technologies and applications related to geospatial data are constantly advancing and updating, so identifying the technologies and applications among them will help foster and fund further innovation. Through topic analysis, new research hotspots can be discovered by understanding the whole development process of a topic. At present, the main methods to determine topics are peer review and bibliometrics, however they just review relevant literature or perform simple frequency analysis. This paper proposes a new topic discovery method, which combines a word embedding method, based on a pre-trained model, Bert, and a spherical k-means clustering algorithm, and applies the similarity between literature and topics to assign literature to different topics. The proposed method was applied to 266 pieces of literature related to geospatial data over the past five years. First, according to the number of publications, the trend analysis of technologies and applications related to geospatial data in several leading countries was conducted. Then, the consistency of the proposed method and the existing method PLSA (Probabilistic Latent Semantic Analysis) was evaluated by using two similar consistency evaluation indicators (i.e., U-Mass and NMPI). The results show that the method proposed in this paper can well reveal text content, determine development trends, and produce more coherent topics, and that the overall performance of Bert-LSA is better than PLSA using NPMI and U-Mass. This method is not limited to trend analysis using the data in this paper; it can also be used for the topic analysis of other types of texts.

Sparse Topic Modeling: Computational Efficiency, Near-Optimal Algorithms, and Statistical Inference

Sparse topic modeling under the probabilistic latent semantic indexing (pLSI) model is studied. Novel and computationally fast algorithms for estimation and inference of both the word-topic matrix and the topic-document matrix are proposed and their theoretical properties are investigated. Both minimax upper and lower bounds are established and the results show that the proposed algorithms are rate-optimal, up to a logarithmic factor. Moreover, a refitting algorithm is proposed to establish asymptotic normality and construct valid confidence intervals for the individual entries of the word-topic and topic-document matrices. Simulation studies are carried out to investigate the numerical performance of the proposed algorithms. The results show that the proposed algorithms perform well numerically and are more accurate in a range of simulation settings comparing to the existing literature. In addition, the methods are illustrated through an analysis of the COVID-19 Open Research Dataset (CORD-19).

Sparse User Check-in Venue Prediction By Exploring Latent Decision Contexts From Location-Based Social Networks

The proliferation of online Location-Based Social Networks (LBSN) has offered unprecedented opportunities for understanding fine-grained spatio-temporal behaviors of users and developing new location-aware applications. In this article, we focus on the problem of “Sparse User Check-in Venue Prediction,” where the goal is to predict the next venue LBSN users will visit by exploiting their sparse online check-in traces and the latent decision contexts. While efforts have been made to predict users’ check-in traces on a LBSN, several important challenges still exist. First, check-in traces contributed by LBSN users are often too sparse to provide sufficient evidence for a reliable prediction, especially when the prediction space is huge (e.g., hundreds of thousands of venues in large cities). Second, the user's decision context on which venue to visit next is often latent and has not been incorporated by current venue prediction models. Third, the dynamic and non-deterministic dependency between check-ins is either ignored or replaced by a simplified “consecutiveness” assumption in existing solutions, leading to sub-optimal prediction results. In this article, we develop a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Context-aware Sparse Check-in Venue Prediction (CSCVP)</i> scheme inspired by natural language processing techniques to address the above challenges. In particular, CSCVP predicts the venue category information and explores the similarity between users to address data sparsity challenge by significantly reducing the prediction space. It also leverages the Probabilistic Latent Semantic Analysis (PLSA) model to incorporate the user decision context into the prediction model. Finally, we develop a novel Temporal Adaptive Ngram (TA-Ngram) model in CSCVP to capture the dynamic and non-deterministic dependency between check-ins. We evaluate CSCVP using three real-world LBSN datasets. The results show that our scheme significantly improves accuracy (30.9 percent improvement) of the state-of-the-art user check-in venue prediction solutions.

A mathematical comparison of non-negative matrix factorization related methods with practical implications for the analysis of mass spectrometry imaging data.

RationaleNon‐negative matrix factorization (NMF) has been used extensively for the analysis of mass spectrometry imaging (MSI) data, visualizing simultaneously the spatial and spectral distributions present in a slice of tissue. The statistical framework offers two related NMF methods: probabilistic latent semantic analysis (PLSA) and latent Dirichlet allocation (LDA), which is a generative model. This work offers a mathematical comparison between NMF, PLSA, and LDA, and includes a detailed evaluation of Kullback–Leibler NMF (KL‐NMF) for MSI for the first time. We will inspect the results for MSI data analysis as these different mathematical approaches impose different characteristics on the data and the resulting decomposition.MethodsThe four methods (NMF, KL‐NMF, PLSA, and LDA) are compared on seven different samples: three originated from mice pancreas and four from human‐lymph‐node tissues, all obtained using matrix‐assisted laser desorption/ionization time‐of‐flight mass spectrometry (MALDI‐TOF MS).ResultsWhere matrix factorization methods are often used for the analysis of MSI data, we find that each method has different implications on the exactness and interpretability of the results. We have discovered promising results using KL‐NMF, which has only rarely been used for MSI so far, improving both NMF and PLSA, and have shown that the hitherto stated equivalent KL‐NMF and PLSA algorithms do differ in the case of MSI data analysis. LDA, assumed to be the better method in the field of text mining, is shown to be outperformed by PLSA in the setting of MALDI‐MSI. Additionally, the molecular results of the human‐lymph‐node data have been thoroughly analyzed for better assessment of the methods under investigation.ConclusionsWe present an in‐depth comparison of multiple NMF‐related factorization methods for MSI. We aim to provide fellow researchers in the field of MSI a clear understanding of the mathematical implications using each of these analytical techniques, which might affect the exactness and interpretation of the results.