Latent Semantic Model Research Articles

Collaborative recommendation algorithm based on probabilistic matrix factorization in probabilistic latent semantic analysis

In order to effectively solve the problem of new items and obviously improve the accuracy of the recommended results, we proposed a collaborative recommendation algorithm based on improved probabilistic latent semantic model in this paper, which introduces popularity factor into probabilistic latent semantic analysis to derive probabilistic matrix factorization model. The core idea is to integrate the semantic knowledge into the recommendation process to overcome the shortcomings of the traditional recommendation algorithm. We introduced popularity factor to form a quintuple vector so as to understand user preference, and can integrate the probabilistic matrix factorization to solve the problem of data sparsity on basis of Probabilistic Latent Semantic Analysis; then the probabilistic matrix factorization model is adopted to construct the weighted similarity function to compute the recommendation result. Experimental study on real-world data-sets demonstrates that our proposed method can outperform three state-of-the art methods in recommendation accuracy.

Music auto-tagging based on the unified latent semantic modeling

We proposed a music auto-tagging approach based on the latent space modeling both for music context and content. First, we introduce the latent semantic analysis for music tags with Sparse Nonnegative Matrix Factorization. Then the music contents semantics will be learnt by decomposing the music content into a pre-trained dictionary and an adaptive dictionary learning algorithm is proposed. Finally, the two latent spaces will be associated with a certain subspace mapping algorithm. The experimental results show that our proposed approach outperforms the state-of-the-art auto-tagging systems when applied to the CAL500 dataset in the 5-fold cross-validation experiments.

Latent semantic extraction and analysis for TRIZ-based inventive design

During the development of TRIZ, several knowledge sources have been developed to solve inventive problems. Even though they are about close notions, the level of detail of the descriptions is very dissimilar, making it difficult for the user to operate with them in a systematic way. To cope with this difficulty, we are interested in finding semantic links among these sources, with the goal of assisting the inventive design expert during his activities. Taking into account that all the TRIZ knowledge sources are represented as short texts and directly applying conventional topic models on such short texts may not work well, an extended latent semantic analysis model is proposed to find the missing links among the TRIZ knowledge sources. With the help of these obtained links, several heuristic abstract solutions can be obtained. In order to show this whole process, the resolution of the case of the 'Auguste Piccard's Stratostat' is elaborated in detail. [Received: 11 March 2017; Revised: 21 November 2017; Accepted 30 March 2018]

Latent Semantic Extraction and Analysis for TRIZ-Based Inventive Design

During the development of TRIZ, several knowledge sources have been developed to solve inventive problems. Even though they are about close notions, the level of detail of the descriptions is very dissimilar, making it difficult for the user to operate with them in a systematic way. To cope with this difficulty, we are interested in finding semantic links among these sources, with the goal of assisting the inventive design expert during his activities. Taking into account that all the TRIZ knowledge sources are represented as short texts and directly applying conventional topic models on such short texts may not work well, an extended latent semantic analysis model is proposed to find the missing links among the TRIZ knowledge sources. With the help of these obtained links, several heuristic abstract solutions can be obtained. In order to show this whole process, the resolution of the case of the 'Auguste Piccard's Stratostat' is elaborated in detail. [Received: 11 March 2017; Revised: 21 November 2017; Accepted 30 March 2018]

Emphasizing Essential Words for Sentiment Classification Based on Recurrent Neural Networks

With the explosion of online communication and publication, texts become obtainable via forums, chat messages, blogs, book reviews and movie reviews. Usually, these texts are much short and noisy without sufficient statistical signals and enough information for a good semantic analysis. Traditional natural language processing methods such as Bow-of-Word (BOW) based probabilistic latent semantic models fail to achieve high performance due to the short text environment. Recent researches have focused on the correlations between words, i.e., term dependencies, which could be helpful for mining latent semantics hidden in short texts and help people to understand them. Long short-term memory (LSTM) network can capture term dependencies and is able to remember the information for long periods of time. LSTM has been widely used and has obtained promising results in variants of problems of understanding latent semantics of texts. At the same time, by analyzing the texts, we find that a number of keywords contribute greatly to the semantics of the texts. In this paper, we establish a keyword vocabulary and propose an LSTM-based model that is sensitive to the words in the vocabulary; hence, the keywords leverage the semantics of the full document. The proposed model is evaluated in a short-text sentiment analysis task on two datasets: IMDB and SemEval-2016, respectively. Experimental results demonstrate that our model outperforms the baseline LSTM by 1%~2% in terms of accuracy and is effective with significant performance enhancement over several non-recurrent neural network latent semantic models (especially in dealing with short texts). We also incorporate the idea into a variant of LSTM named the gated recurrent unit (GRU) model and achieve good performance, which proves that our method is general enough to improve different deep learning models.

GitHub API Based QuantNet Mining Infrastructure in R

QuantNet being an online GitHub based organization is an integrated environment consisting of different types of statistics-related documents and program codes called Quantlets. The QuantNet Style Guide and the yamldebugger package allow a standardized audit and validation of YAML annotated software repositories within this organization. The behavior statistics of QuantNet users are measured with Web Metrics from Google Analytics. We show how the search queries obtained from Google’s metrics can be used in the test collections in order to calibrate and evaluate the information retrieval (IR) performance of QuantNet’s search engine called QuantNetXploRer. For that purpose, different text mining (TM) models will be examined by means of the new TManalyzer package. Further, we introduce the Validation Pipeline (Vali-PP) and apply it on the YAML data. Vali-PP is a functional multi-staged instrument for clustering analysis, providing multivariate statistical analysis of the co-occurrence distribution of driving factors of the pipeline. The new package rgithubS, which enables a GitHub wide search for code and repositories using the GitHub Search API and which is an essential element of the QuantNet Mining infrastructure, is briefly presented. The TManalyzer results show that for all considered single term queries the number of true positives is maximal in a latent semantic analysis model configuration (LSA50). The Vali-PP analysis indicates that the optimality of the combination LSA50 and hierarchical clustering (HC) applies to 70 − 90% of the cluster sizes for most of the considered quality indices. Further, we can infer that more accurate and comprehensive metadata increases the clustering quality. Subsequently, the findings of our experimental design are implemented into the QuantNetXploRer. The GitHub API driven QuantNetXploRer can be found and mined under http://www.quantlet.de

Salience Estimation via Variational Auto-Encoders for Multi-Document Summarization

We propose a new unsupervised sentence salience framework for Multi-Document Summarization (MDS), which can be divided into two components: latent semantic modeling and salience estimation. For latent semantic modeling, a neural generative model called Variational Auto-Encoders (VAEs) is employed to describe the observed sentences and the corresponding latent semantic representations. Neural variational inference is used for the posterior inference of the latent variables. For salience estimation, we propose an unsupervised data reconstruction framework, which jointly considers the reconstruction for latent semantic space and observed term vector space. Therefore, we can capture the salience of sentences from these two different and complementary vector spaces. Thereafter, the VAEs-based latent semantic model is integrated into the sentence salience estimation component in a unified fashion, and the whole framework can be trained jointly by back-propagation via multi-task learning. Experimental results on the benchmark datasets DUC and TAC show that our framework achieves better performance than the state-of-the-art models.

Semantically Enhanced Medical Information Retrieval System: A Tensor Factorization Based Approach

Medical information retrieval plays an increasingly important role to help physicians and domain experts to better access medical-related knowledge and information, and support decision making. Integrating the medical knowledge bases has the potential to improve the information retrieval performance through incorporating medical domain knowledge for relevance assessment. However, this is not a trivial task due to the challenges to effectively utilize the domain knowledge in the medical knowledge bases. In this paper, we proposed a novel medical information retrieval system with a two-stage query expansion strategy, which is able to effectively model and incorporate the latent semantic associations to improve the performance. This system consists of two parts. First, we applied a heuristic approach to enhance the widely used pseudo relevance feedback method for more effective query expansion, through iteratively expanding the queries to boost the similarity score between queries and documents. Second, to improve the retrieval performance with structured knowledge bases, we presented a latent semantic relevance model based on tensor factorization to identify semantic association patterns under sparse settings. These identified patterns are then used as inference paths to trigger knowledge-based query expansion in medical information retrieval. Experiments with the TREC CDS 2014 data set: 1) showed that the performance of the proposed system is significantly better than the baseline system and the systems reported in TREC CDS 2014 conference, and is comparable with the state-of-the-art systems and 2) demonstrated the capability of tensor-based semantic enrichment methods for medical information retrieval tasks.

Eye movements in software traceability link recovery

Information Retrieval (IR) approaches, such as Latent Semantic Indexing (LSI) and Vector Space Model (VSM), are commonly applied to recover software traceability links. Recently, an approach based on developers' eye gazes was proposed to retrieve traceability links. This paper presents a comparative study on IR and eye-gaze based approaches. In addition, it reports on the possibility of using eye gaze links as an alternative benchmark in comparison to commits. The study conducted asked developers to perform bug-localization tasks on the open source subject system JabRef. The iTrace environment, which is an eye tracking enabled Eclipse plugin, was used to collect eye gaze data. During the data collection phase, an eye tracker was used to gather the source code entities (SCE's), developers looked at while solving these tasks. We present an algorithm that uses the collected gaze dataset to produce candidate traceability links related to the tasks. In the evaluation phase, we compared the results of our algorithm with the results of an IR technique, in two different contexts. In the first context, precision and recall metric values are reported for both IR and eye gaze approaches based on commits. In the second context, another set of developers were asked to rate the candidate links from each of the two techniques in terms of how useful they were in fixing the bugs. The eye gaze approach outperforms standard LSI and VSM approaches and reports a 55 % precision and 67 % recall on average for all tasks when compared to how the developers actually fixed the bug. In the second context, the usefulness results show that links generated by our algorithm were considered to be significantly more useful (to fix the bug) than those of the IR technique in a majority of tasks. We discuss the implications of this radically different method of deriving traceability links. Techniques for feature location/bug localization are commonly evaluated on benchmarks formed from commits as is done in the evaluation phase of this study. Although, commits are a reasonable source, they only capture entities that were eventually changed to fix a bug or resolve a feature. We investigate another type of benchmark based on eye tracking data, namely links generated from the bug-localization tasks given to the developers in the data collection phase. The source code entities relevant to subjected bugs recommended from IR methods are evaluated on both commits and links generated from eye gaze. The results of the benchmarking phase show that the use of eye tracking could form an effective (complementary) benchmark and add another interesting perspective in the evaluation of bug-localization techniques.

Land Use detection with cell phone data using topic models: Case Santiago, Chile

Today we have the opportunity without precedents to analyze human land use or mobility behavior in a city, country or even the globe. Some studies have analyzed existing data generated daily by mobile networks, mostly using geo-localization in Twitter, Foursquare or cell phone records. Most of these studies use a small portion of data (a few days or a couple million records). This time we will show a novel way to apply latent semantic topic models to detect Land Use Patterns in a real big dataset of 880,000,000 calls made in Santiago City (Chile) over 77days by about 3 million customers of a major telecommunications company. We proposed to use a latent variables clustering technique which allow us to detect four interesting clusters. We found out that the application of LDA allow us to discover two well known clusters (residential and office area clusters) but also we discover two new clusters: Leisure-Commerce and Rush Hour patterns.

Complex human activities recognition using interval temporal syntactic model

A novel method based on interval temporal syntactic model was proposed to recognize human activities in video flow. The method is composed of two parts: feature extract and activities recognition. Trajectory shape descriptor, speeded up robust features (SURF) and histograms of optical flow (HOF) were proposed to represent human activities, which provide more exhaustive information to describe human activities on shape, structure and motion. In the process of recognition, a probabilistic latent semantic analysis model (PLSA) was used to recognize sample activities at the first step. Then, an interval temporal syntactic model, which combines the syntactic model with the interval algebra to model the temporal dependencies of activities explicitly, was introduced to recognize the complex activities with a time relationship. Experiments results show the effectiveness of the proposed method in comparison with other state-of-the-art methods on the public databases for the recognition of complex activities.

A study of user profile representation for personalized cross-language information retrieval

Purpose– With an increase in the amount of multilingual content on the World Wide Web, users are often striving to access information provided in a language of which they are non-native speakers. The purpose of this paper is to present a comprehensive study of user profile representation techniques and investigate their use in personalized cross-language information retrieval (CLIR) systems through the means of personalized query expansion.Design/methodology/approach– The user profiles consist of weighted terms computed by using frequency-based methods such as tf-idf and BM25, as well as various latent semantic models trained on monolingual documents and cross-lingual comparable documents. This paper also proposes an automatic evaluation method for comparing various user profile generation techniques and query expansion methods.Findings– Experimental results suggest that latent semantic-weighted user profile representation techniques are superior to frequency-based methods, and are particularly suitable for users with a sufficient amount of historical data. The study also confirmed that user profiles represented by latent semantic models trained on a cross-lingual level gained better performance than the models trained on a monolingual level.Originality/value– Previous studies on personalized information retrieval systems have primarily investigated user profiles and personalization strategies on a monolingual level. The effect of utilizing such monolingual profiles for personalized CLIR remains unclear. The current study fills the gap by a comprehensive study of user profile representation for personalized CLIR and a novel personalized CLIR evaluation methodology to ensure repeatable and controlled experiments can be conducted.

An empirical comparison of latent sematic models for applications in industry

In recent years, topic models have been gaining popularity to perform classification of text from several web sources (from social networks to digital media). However, after working for many years in the web text mining area we have notice that assessing the quality of topics discovered is still an open problem, quite hard to solve. In this paper, we evaluated four latent semantic models using two metrics: coherence and interpretability which are the most used. We show how these pure mathematical metrics fall short to asses topics quality. Experiments were performed over a dataset of 21,863 text reclamation.

Research on Perception-Oriented Image Scene and Emotion Categorization

With the development of multimedia technology and computer network, the number of available images in- creases with an explosive speed. But the technology also brings some trouble to its users, sometimes it's very difficult for us to find some details that very important from a huge amount of available data. At this time, image scene and emotion categorization technologies are required urgently. The purpose of emotional image classification is that we hope the com- puter can express the emotion reaction when observing the image, and classify the images into the different emotional categories automatically. The process of scene image classification is that how to make computer systems to classify the image sets automatically which contain semantic information, according to the visual perception mechanism of human. Here, for the scene categorization problem based on the visual words, the dissertation presents a novel learning framework to design discriminating semantic visual words. At last, for the emotion categorization of natural scene images, the disser- tation presents an emotion categorization using Affective-probabilistic Latent Semantic Analysis model based on the vis- ual cognitive theory.

Using Twitter data to predict the performance of Bollywood movies

Purpose– The purpose of this paper is to address the shortcomings of limited research in forecasting the power of social media in India.Design/methodology/approach– This paper uses sentiment analysis and prediction algorithms to analyze the performance of Indian movies based on data obtained from social media sites. The authors used Twitter4j Java API for extracting the tweets through authenticating connection with Twitter web sites and stored the extracted data in MySQL database and used the data for sentiment analysis. To perform sentiment analysis of Twitter data, the Probabilistic Latent Semantic Analysis classification model is used to find the sentiment score in the form of positive, negative and neutral. The data mining algorithm Fuzzy Inference System is used to implement sentiment analysis and predict movie performance that is classified into three categories: hit, flop and average.Findings– In this study the authors found results of movie performance at the box office, which had been based on fuzzy interface system algorithm for prediction. The fuzzy interface system contains two factors, namely, sentiment score and actor rating to get the accurate result. By calculation of opening weekend collection, the authors found that that the predicted values were approximately same as the actual values. For the movieSingham Returnsover method of prediction gave a box office collection as 84 crores and the actual collection turned out to be 88 crores.Research limitations/implications– The current study suffers from the limitation of not having enough computing resources to crawl the data. For predicting box office collection, there is no correct availability of ticket price information, total number of seats per screen and total number of shows per day on all screens. In the future work the authors can add several other inputs like budget of movie, Central Board of Film Certification rating, movie genre, target audience that will improve the accuracy and quality of the prediction.Originality/value– The authors used different factors for predicting box office movie performance which had not been used in previous literature. This work is valuable for promoting of product and services of the firms.

Preference analysis of verbal protocol and drawned picture using latent semantic analysis model

Preference analysis of verbal protocol and drawned picture using latent semantic analysis model

The potential relationship discovery model based on result fusion for biomedical medicine research

With the amount of biomedical data growing explosively, medical scientists use many datasets in research for new medicine. However, the amount of biomedical data is growing too fast to abstract hidden information. At the same time, with the development of data storage diversification, scientists prefer to have data fusion based on heterogeneous data sources as opposed to a single data source, and ultimately to achieve knowledge and discovery across heterogeneous databases. Our study focuses on extending the application of latent semantic analysis methodologies into the area of biomedical research. Our purpose is to develop a model for discovering potential relationships between medicines and diseases based on biomedical latent semantic analysis. This model could be used in constructing link maps for biomedical entities, and provide a theoretical basis and practical support for biomedical scientists in their study of the disease–medicine relationship. In detail, we discuss the study of the integration of the latent semantic analysis model and data fusion methodologies. Our result fusion solution combines scientific literature repositories and a biomedical database based on context and the ABC model, and is supervised by a semi-supervised learning algorithm and data fusion algorithms. The expectation is that fused data could represent multilevel potential relationships between biological entities and related emotional relationship expression. The model is validated by experience and proven to be feasible and effective.

Latent Semantic Rational Kernels for Topic Spotting on Conversational Speech

In this work, we propose latent semantic rational kernels (LSRK) for topic spotting on conversational speech. Rather than mapping the input weighted finite-state transducers (WFSTs) onto a high dimensional n-gram feature space as in n-gram rational kernels, the proposed LSRK maps the WFSTs onto a latent semantic space. With the proposed LSRK, all available external knowledge and techniques can be flexibly integrated into a unified WFST based framework to boost the topic spotting performance. We present how to generalize the LSRK using tf-idf weighting, latent semantic analysis, WordNet and probabilistic topic models. To validate the proposed LSRK framework, we conduct the topic spotting experiments on two datasets, Switchboard and AT&T HMIHY0300 initial collection. The experimental results show that with the proposed LSRK we can achieve significant and consistent topic spotting performance gains over the n-gram rational kernels.

Scene recognition by semantic visual words

In this paper, we propose a novel approach to introduce semantic relations into the bag-of-words framework. We use the latent semantic models, such as latent semantic analysis (LSA) and probabilistic latent semantic analysis (pLSA), in order to define semantically rich features and embed the visual features into a semantic space. The semantic features used in LSA technique are derived from the low-rank approximation of word–image occurrence matrix by singular value decomposition. Similarly, by using the pLSA approach, the topic-specific distributions of words can be considered dimensions of a concept space. In the proposed space, the distances between words represent the semantic distances which are used for constructing a discriminative and semantically meaningful vocabulary. Position information significantly improves scene recognition accuracy. Inspired by this, in this paper, we bring position information into the proposed semantic vocabulary frameworks. We have tested our approach on the 15-Scene and 67-MIT Indoor datasets and have achieved very promising results.

Learning latent semantic model with visual consistency for image analysis

Latent semantic models (e.g. PLSA and LDA) have been successfully used in document analysis. In recent years, many of the latent semantic models have also been proved to be promising for visual con...