Pachinko Allocation Model Research Articles

TransVAE-PAM: A Combined Transformer and DAG-based Approach for Enhanced Fake News Detection in Indian Context

In this study, we introduce a novel method, “TransVAE-PAM”, for the classification of fake news articles, tailored specifically for the Indian context. The approach capitalizes on state-of-the-art contextual and sentence transformer-based embedding models to generate article embeddings. Furthermore, we also try to address the issue of compact model size. In this respect, we employ a Variational Autoencoder (VAE) and β -VAE to reduce the dimensions of the embeddings, thereby yielding compact latent representations. To capture the thematic essence or important topics in the news articles, we use the Pachinko Allocation Model (PAM) model, a Directed Acyclic Graph (DAG) based approach, to generate meaningful topics. These two facets of representation - the reduced-dimension embeddings from the VAE and the extracted topics from the PAM model - are fused together to create a feature set. This representation is subsequently channeled into five different methods for fake news classification. Furthermore, we use eight distinct transformer-based architectures to test the embedding generation. To validate the feasibility of the proposed approach, we have conducted extensive experimentation on a proprietary dataset. The dataset is sourced from “Times of India” and other online media. Considering the size of the dataset, large-scale experiments are conducted on an NVIDIA supercomputer. Through this comprehensive numerical investigation, we have achieved an accuracy of 96.2% and an F1 score of 96% using the DistilBERT transformer architecture. By complementing the method via topic modeling, we record a performance improvement with the accuracy and F1 score both at 97%. These results indicate a promising direction toward leveraging the combination of advanced topic models into existing classification schemes to enhance research on fake news detection.

Topic Modeling Techniques for Text Mining Over a Large-Scale Scientific and Biomedical Text Corpus

Topic models are efficient in extracting central themes from large-scale document collection and it is an active research area. The state-of-the-art techniques like Latent Dirichlet Allocation, Correlated Topic Model (CTM), Hierarchical Dirichlet Process (HDP), Dirichlet Multinomial Regression (DMR) and Hierarchical Pachinko Allocation (HPA) model is considered for comparison. . The abstracts of articles were collected between different periods from PUBMED library by keywords adolescence substance use and depression. A lot of research has happened in this area and thousands of articles are available on PubMed in this area. This collection is huge and so extracting information is very time-consuming. To fit the topic models this extracted text data is used and fitted models were evaluated using both likelihood and non-likelihood measures. The topic models are compared using the evaluation parameters like log-likelihood and perplexity. To evaluate the quality of topics topic coherence measures has been used.

Analysis and tuning of hierarchical topic models based on Renyi entropy approach.

Hierarchical topic modeling is a potentially powerful instrument for determining topical structures of text collections that additionally allows constructing a hierarchy representing the levels of topic abstractness. However, parameter optimization in hierarchical models, which includes finding an appropriate number of topics at each level of hierarchy, remains a challenging task. In this paper, we propose an approach based on Renyi entropy as a partial solution to the above problem. First, we introduce a Renyi entropy-based metric of quality for hierarchical models. Second, we propose a practical approach to obtaining the “correct” number of topics in hierarchical topic models and show how model hyperparameters should be tuned for that purpose. We test this approach on the datasets with the known number of topics, as determined by the human mark-up, three of these datasets being in the English language and one in Russian. In the numerical experiments, we consider three different hierarchical models: hierarchical latent Dirichlet allocation model (hLDA), hierarchical Pachinko allocation model (hPAM), and hierarchical additive regularization of topic models (hARTM). We demonstrate that the hLDA model possesses a significant level of instability and, moreover, the derived numbers of topics are far from the true numbers for the labeled datasets. For the hPAM model, the Renyi entropy approach allows determining only one level of the data structure. For hARTM model, the proposed approach allows us to estimate the number of topics for two levels of hierarchy.

Descriptive Naming & Summarization of large text using Topic Model-A Survey

Due to the plethora of documents containing large scale of text that are available on web it sometimes gets difficult to go through each document to get the clear picture of what the text is depicting. In this paper, we are analyzing several techniques to evaluate Topic Model. A Topic Model is a very popular approach for representing and smoothing the content of documents. Here we will focus on uncovering the thematic structure of a corpus of document that will help in document classification and for compact document topic representation. We have gone through some of the famous topic model such as-Latent Semantic Indexing (LSI),Probabilistic Latent Semantic Indexing (PLSI),Latent Dirichlet Allocation (LDA),Pachinko Allocation Model (PAM) where we encounter few issues such as Topic models are not proper for some SNS such as micro blogging and supervise learning techniques are designed for one-labeled corpus-i. e. they are limiting the document to a single label.

A Tool to Extract Name Entity Recognition From Big Data in Banking Sectors

Generally, the Internet is the global system of interconnected computer networks, connecting millions of computers as well as people, and thus generates a massive quantity of information on a daily basis. This leads to extracting the necessary information using information filtering (IF) in several domains. In our implementation, the named entity recognition (NER) technique is employed to automatically extract valuable data from the unstructured natural language texts. As several works has been outlined in detecting named entities, plenty of very different NER tools exist for several domains. However, NER remains a giant challenge so to solve this problem we proposed a novel framework by combining three efficient classifiers. This article proposes a three-layered neural network approach with conditional random field (CRF), the Pachinko allocation model (PAM), and the Adaptive Neuro-Fuzzy Inference System (ANFIS) for detecting named entities in three steps. First, a classifier based on CRF is employed to train the input file. Second, PAM is employed to boost the previous output created by CRF to enhance the label annotation. Third, the ANFIS captures the deep features of the information by itself from the pre-trained information to attain accurate predictions. Experimental results show that the learned model yields a banking domain with a recall rate of 92%, a precision rate of 95% and F-measure of 92% by implementing it in the R Platform.

Hierarchical topic modeling with pose-transition feature for action recognition using 3D skeleton data

Despite impressive achievements in image processing and artificial intelligence in the past decade, understanding video-based action remains a challenge. However, the intensive development of 3D computer vision in recent years has brought more potential research opportunities in pose-based action detection and recognition. Thanks to the advantages of depth camera devices like the Microsoft Kinect sensor, we developed an effective approach to in-depth analysis of indoor actions using skeleton information, in which skeleton-based feature extraction and topic model-based learning are two major contributions. Geometric features, i.e. joint distance, joint angle, and joint-plane distance are calculated in the spatio-temporal dimension. These features are merged into two types, called pose and transition features, and then are provided to codebook construction to convert sparse features into visual words by k-means clustering. An efficient hierarchical model is developed to describe the full correlation of feature - poselet - action based on Pachinko Allocation Model. This model has the potential to uncover more hidden poselets, which have been recognized as the valuable information and help to differentiate pose-sharing actions. The experimental results on several well-known datasets, such as MSR Action 3D, MSR Daily Activity 3D, Florence 3D Action, UTKinect-Action 3D, and NTU RGB+D Action Recognition, demonstrate the high recognition accuracy of the proposed method. Our method outperforms state-of-the-art methods in the field in most dataset benchmarks.

Content-based image retrieval with pachinko allocation model and a combination of colour, texture and text features

We present in this paper a content-based image retrieval system (CBIR) based on pachinko allocation model (PAM) and employing a combination of colour, texture and textual features. PAM is a probabilistic topic model which captures correlation not only between words in documents but also between different topics (concepts) responsible of their generation. Regardless this advantage of PAM, there is no work which explores its utility for content based image retrieval tasks. We aim to evaluate the use of PAM for CBIRs by implementing a system based on it. In this context, PAM was applied with two different modalities of features, image global features and textual indexes separately and combined. Mean average precision is evaluated. The use of PAM with combination of features has slightly improved results of using it with just one modality, this opens more perspective in order to enhance results.

Content-based image retrieval with pachinko allocation model and a combination of colour, texture and text features

We present in this paper a content-based image retrieval system (CBIR) based on pachinko allocation model (PAM) and employing a combination of colour, texture and textual features. PAM is a probabilistic topic model which captures correlation not only between words in documents but also between different topics (concepts) responsible of their generation. Regardless this advantage of PAM, there is no work which explores its utility for content based image retrieval tasks. We aim to evaluate the use of PAM for CBIRs by implementing a system based on it. In this context, PAM was applied with two different modalities of features, image global features and textual indexes separately and combined. Mean average precision is evaluated. The use of PAM with combination of features has slightly improved results of using it with just one modality, this opens more perspective in order to enhance results.

Extracting Hierarchy of Coherent User-Concerns to Discover Intricate User Behavior from User Reviews

Intricate user-behaviors can be understood by discovering user interests from their reviews. Topic modeling techniques have been extensively explored to discover latent user interests from user reviews. However, a topic extracted by topic modelling techniques can be a mixture of several quite different concepts and thus less interpretable. In this paper, the authors present a method that uses topic modeling techniques to discover a large number of topics and applies hierarchical clustering to generate a much smaller number of interpretable User-Concerns. These User-Concerns are further compared with topics generated by Latent Dirichlet Allocation (LDA) and Pachinko Allocation Model (PAM) and shown to be more coherent and interpretable. The authors cut the linkage tree formed while performing the hierarchical clustering of the User-Concerns, at different levels, and generate a hierarchy of User-Concerns. They also discuss how collaborative filtering based recommendation systems can be enriched by infusing additional user-behavioral knowledge from such hierarchy.

Interactive activity recognition using pose-based spatio–temporal relation features and four-level Pachinko Allocation Model

In this paper, we go beyond the problem of recognizing video-based human interactive activities. We propose a novel approach that permits to deeply understand complex person-person activities based on the knowledge coming from human pose analysis. The joint coordinates of interactive objects are first located by an efficient human pose estimation algorithm. The relation features consisting of the intra and inter-person features of joint distance and angle, are suggested to use for describing the relationships between body components of the individual persons and the interacting two participants in the spatio-temporal dimension. These features are then provided to the codebook construction process, in which two types of codeword are generated corresponding to distance and angle features. In order to explain the relationships between poses, a flexible hierarchical topic model constructed by four layers is proposed using the Pachinko Allocation Model. The model is able to represent the full correlation between the relation features of body components as codewords, the interactive poselets as subtopics, and the interactive activities as super topics. Discrimination of complex activities presenting similar postures is further obtained by the proposed model. We validate our interaction recognition method on two practical data sets, the BIT-Interaction data set and the UT-Interaction data set. The experimental results demonstrate that the proposed approach outperforms recent interaction recognition approaches in terms of recognition accuracy.

Traffic Behavior Recognition Using the Pachinko Allocation Model.

CCTV-based behavior recognition systems have gained considerable attention in recent years in the transportation surveillance domain for identifying unusual patterns, such as traffic jams, accidents, dangerous driving and other abnormal behaviors. In this paper, a novel approach for traffic behavior modeling is presented for video-based road surveillance. The proposed system combines the pachinko allocation model (PAM) and support vector machine (SVM) for a hierarchical representation and identification of traffic behavior. A background subtraction technique using Gaussian mixture models (GMMs) and an object tracking mechanism based on Kalman filters are utilized to firstly construct the object trajectories. Then, the sparse features comprising the locations and directions of the moving objects are modeled by PAM into traffic topics, namely activities and behaviors. As a key innovation, PAM captures not only the correlation among the activities, but also among the behaviors based on the arbitrary directed acyclic graph (DAG). The SVM classifier is then utilized on top to train and recognize the traffic activity and behavior. The proposed model shows more flexibility and greater expressive power than the commonly-used latent Dirichlet allocation (LDA) approach, leading to a higher recognition accuracy in the behavior classification.

Boosting scene understanding by hierarchical pachinko allocation

Scene understanding is a popular research direction. In this area, many attempts focus on the problem of naming objects in the complex natural scene, and visual semantic integration model (VSIM) is the representative. This model consists of two parts: semantic level and visual level. In the first level, it uses a four-level pachinko allocation model (PAM) to capture the semantics behind images. However, this four-level PAM is inflexible and lacks of considerations of common subtopics that represent the background semantics. To address these problems, we use hierarchical PAM (hPAM) to replace PAM. Since hPAM is flexible, we investigate two variations of hPAM to boost VSIM in this paper. We derive the Gibbs sampler to learn the proposed models. Empirical results validate that our works can obtain better performance than the state-of-the-art algorithms.

Microblog Hot Spot Mining Based on PAM Probabilistic Topic Model

Microblogs are short texts carried with limited information, which will increase the difficulty of topic mining. This paper proposes the use of PAM (Pachinko Allocation Model) probabilistic topic model to extract the generative model of text’s implicit theme for microblog hot spot mining. First, three categories of microblog and the main contribution of this paper are illustrated. Second, for there are four topic models which are respectively explained, the PAM model is introduced in detail in terms of how to generate a document, the accuracy of document classification and the topic correlation in PAM. Finally, MapReduce is described. For the number of microblogs is huge as well as the number of contactors, the totally number of words is relatively small. With MapReduce, microblogs data are split by contactor, document-topic count matrix and contactor-topic count matrix can be locally stored while the word-topic count matrix must be globally stored. Thus, the hot spot mining can be achieved on the basis of PAM probabilistic topic model.

A PAM-based ontology concept and hierarchy learning method

As a shared conceptual model that can express knowledge and a modelling tool that can describe conceptual model in the semantic and knowledge level, ontology plays an important role in the related fields of the Semantic Web, natural language processing and information retrieval. Ontology learning is a series of methods and technologies to construct ontology automatically or semi-automatically. Concept and hierarchy learning are the most important parts of the ontology construction. This paper proposes an ontology concept and hierarchy learning method based on the Pachinko Allocation Model. The above problem is transformed into a probability and statistical inference problem by building an ontology concept learning model. Gibbs sampling is used to estimate the parameters. Then, using the ontology concept generation algorithm based on WordNet, an abstract description of the ontology concept is obtained. Experimental results on the standard test dataset show that the proposed method can offer an effective solution to ontology concept and hierarchy learning.

Capturing correlations of multiple labels: A generative probabilistic model for multi-label learning

Recent years have witnessed a considerable surge of interest in the multi-label learning problem. It has been shown that a key factor for a successful multi-label learning algorithm is to effectively exploit relations between labels. However, most of the previous work exploiting label relations focuses on pairwise relations. To handle the situations where there are intrinsic correlations among multiple labels, in this paper, we propose a generative model, Labeled Four-Level Pachinko Allocation Model (L-F-L-PAM), to capture correlations among multiple labels. In our approach of multi-label learning on text data, we apply the proposed model for inferring the training data and the standard Four-Level Pachinko Allocation Model for the test data. Furthermore, we propose a pruned Gibbs Sampling algorithm in the test stage to reduce the inference time. Finally, extensive experiments have been performed to validate the effectiveness and efficiency of our new approach. The results demonstrate significant improvements of our model over Labeled LDA (L-LDA) and superiority in terms of both effectiveness and computational efficiency over other high-performing multi-label learning methods.