Latent Semantics Research Articles

Scene classification based on multifeature probabilistic latent semantic analysis for high spatial resolution remote sensing images

Scene classification can mine high-level semantic information scene categories from low-level visual features for high spatial resolution remote sensing images (HSRIs). A multifeature probabilistic latent semantic analysis (MPLSA) algorithm is proposed to perform the task of scene classification for HSRIs. Distinct from the traditional probabilistic latent semantic analysis (PLSA) with a single feature, to utilize the spatial information of the HSRIs, in MPLSA, multiple features, including spectral and texture features, and the scale-invariant feature transform feature, are combined with PLSA. The visual words are characterized by the multifeature descriptor, and an image set is represented by a discriminative word-image matrix. During the training phase, the MPLSA model mines the visual words’ latent semantics. For unknown images, the MPLSA model analyzes their corresponding latent semantic distributions by combining the words’ latent semantics obtained from the training step. The spectral angle mapper classifier is utilized to label the scene class, based on the image’s latent semantic distribution. The experimental results demonstrate that the proposed MPLSA method can achieve better scene classification accuracy than the traditional single-feature PLSA method.

Generating query suggestions by exploiting latent semantics in query logs

Search engines assist users in expressing their information needs more accurately by reformulating the issued queries automatically and suggesting the generated formulations to the users. Many approaches to query suggestion draw on the information stored in query logs, recommending recorded queries that are textually similar to the current user’s query or that frequently co-occurred with it in the past. In this paper, we propose an approach that concentrates on deducing the actual information need from the user’s query. The challenge therein lies not only in processing keyword queries, which are often short and possibly ambiguous, but especially in handling the complexity of natural language that allows users to express the same or similar information needs in various differing ways. We expect a higher-level semantic representation of a user’s query to more accurately reflect the information need than the explicit query terms alone can. To this aim, we employ latent Dirichlet allocation as a probabilistic topic model to reveal latent semantics in the query log. Our evaluations show that, whereas purely topic-based query suggestion performs the worst, the interpolation of our proposed topic-based model with the baseline word-based model that generates suggestions based on matching query terms achieves significant improvements in suggestion quality over the already well performing purely word-based approach.

Latent semantics of action verbs reflect phonetic parameters of intensity and emotional content.

Conjuring up our thoughts, language reflects statistical patterns of word co-occurrences which in turn come to describe how we perceive the world. Whether counting how frequently nouns and verbs combine in Google search queries, or extracting eigenvectors from term document matrices made up of Wikipedia lines and Shakespeare plots, the resulting latent semantics capture not only the associative links which form concepts, but also spatial dimensions embedded within the surface structure of language. As both the shape and movements of objects have been found to be associated with phonetic contrasts already in toddlers, this study explores whether articulatory and acoustic parameters may likewise differentiate the latent semantics of action verbs. Selecting 3 × 20 emotion-, face-, and hand-related verbs known to activate premotor areas in the brain, their mutual cosine similarities were computed using latent semantic analysis LSA, and the resulting adjacency matrices were compared based on two different large scale text corpora: HAWIK and TASA. Applying hierarchical clustering to identify common structures across the two text corpora, the verbs largely divide into combined mouth and hand movements versus emotional expressions. Transforming the verbs into their constituent phonemes, and projecting them into an articulatory space framed by tongue height and formant frequencies, the clustered small and large size movements appear differentiated by front versus back vowels corresponding to increasing levels of arousal. Whereas the clustered emotional verbs seem characterized by sequences of close versus open jaw produced phonemes, generating up- or downwards shifts in formant frequencies that may influence their perceived valence. Suggesting, that the latent semantics of action verbs reflect parameters of intensity and emotional polarity that appear correlated with the articulatory contrasts and acoustic characteristics of phonemes.

An ontology evolution method based on folksonomy

Folksonomies are increasingly adopted in web systems. These “social taxonomies”, which emerge from collaborative tagging, contrast with the formalism and the systematic creation process applied to ontologies. However, they can play complementary roles, as the knowledge systematically formalized in ontologies by a restricted group can be enriched by the implicit knowledge collaboratively produced by a much wider group. Existing initiatives that involve folksonomies and ontologies are often unidirectional, i.e., ontologies improve tag operations or tags are used to automatically create ontologies. We propose a new fusion approach in which the semantics travels in both directions from folksonomies to ontologies and vice versa. The result of this fusion is our Folksonomized Ontology (FO). In this paper, we present our 3E Steps technique (Extraction, Enrichment, and Evolution), which explores the latent semantics of a given folksonomy (expressed in a FO) to support ontology review and enhancement. It was implemented and tested in a visual review/enhancement tool. All Rights Reserved © 2015 Universidad Nacional Autonoma de Mexico, Centro de Ciencias Aplicadas y Desarrollo Tecnologico. This is an open access item distributed under the Creative Commons CC License BY-NC-ND 4.0.

Latent semantics in language models

This paper investigates three different sources of information and their integration into language modelling. Global semantics is modelled by Latent Dirichlet allocation and brings long range dependencies into language models. Word clusters given by semantic spaces enrich these language models with short range semantics. Finally, our own stemming algorithm is used to further enhance the performance of language modelling for inflectional languages.Our research shows that these three sources of information enrich each other and their combination dramatically improves language modelling. All investigated models are acquired in a fully unsupervised manner.We show the efficiency of our methods for several languages such as Czech, Slovenian, Slovak, Polish, Hungarian, and English, proving their multilingualism. The perplexity tests are accompanied by machine translation tests that prove the ability of the proposed models to improve the performance of a real-world application.

Latent semantics in Named Entity Recognition

In this paper, we propose new features for Named Entity Recognition (NER) based on latent semantics. Furthermore, we explore the effect of unsupervised morphological information on these methods and on the NER system in general. The newly created NER system is fully language-independent thanks to the unsupervised nature of the proposed features. We evaluate the system on English, Spanish, Dutch and Czech corpora and study the difference between weakly and highly inflectional languages. Our system achieves the same or even better results than state-of-the-art language dependent systems. The proposed features proved to be very useful and are the main reason of our promising results.

Integrating Social and Auxiliary Semantics for Multifaceted Topic Modeling in Twitter

Microblogging platforms, such as Twitter, have already played an important role in recent cultural, social and political events. Discovering latent topics from social streams is therefore important for many downstream applications, such as clustering, classification or recommendation. However, traditional topic models that rely on the bag-of-words assumption are insufficient to uncover the rich semantics and temporal aspects of topics in Twitter. In particular, microblog content is often influenced by external information sources, such as Web documents linked from Twitter posts, and often focuses on specific entities, such as people or organizations. These external sources provide useful semantics to understand microblogs and we generally refer to these semantics as auxiliary semantics . In this article, we address the mentioned issues and propose a unified framework for Multifaceted Topic Modeling from Twitter streams. We first extract social semantics from Twitter by modeling the social chatter associated with hashtags. We further extract terms and named entities from linked Web documents to serve as auxiliary semantics during topic modeling. The Multifaceted Topic Model (MfTM) is then proposed to jointly model latent semantics among the social terms from Twitter, auxiliary terms from the linked Web documents and named entities. Moreover, we capture the temporal characteristics of each topic. An efficient online inference method for MfTM is developed, which enables our model to be applied to large-scale and streaming data. Our experimental evaluation shows the effectiveness and efficiency of our model compared with state-of-the-art baselines. We evaluate each aspect of our framework and show its utility in the context of tweet clustering.

Toward Personalized Context Recognition for Mobile Users

The problem of mobile context recognition targets the identification of semantic meaning of context in a mobile environment. This plays an important role in understanding mobile user behaviors and thus provides the opportunity for the development of better intelligent context-aware services. A key step of context recognition is to model the personalized contextual information of mobile users. Although many studies have been devoted to mobile context modeling, limited efforts have been made on the exploitation of the sequential and dependency characteristics of mobile contextual information. Also, the latent semantics behind mobile context are often ambiguous and poorly understood. Indeed, a promising direction is to incorporate some domain knowledge of common contexts, such as “waiting for a bus” or “having dinner,” by modeling both labeled and unlabeled context data from mobile users because there are often few labeled contexts available in practice. To this end, in this article, we propose a sequence-based semisupervised approach to modeling personalized context for mobile users. Specifically, we first exploit the Bayesian Hidden Markov Model (B-HMM) for modeling context in the form of probabilistic distributions and transitions of raw context data. Also, we propose a sequential model by extending B-HMM with the prior knowledge of contextual features to model context more accurately. Then, to efficiently learn the parameters and initial values of the proposed models, we develop a novel approach for parameter estimation by integrating the Dirichlet Process Mixture (DPM) model and the Mixture Unigram (MU) model. Furthermore, by incorporating both user-labeled and unlabeled data, we propose a semisupervised learning-based algorithm to identify and model the latent semantics of context. Finally, experimental results on real-world data clearly validate both the efficiency and effectiveness of the proposed approaches for recognizing personalized context of mobile users.

Group topic model: organizing topics into groups

Latent Dirichlet allocation defines hidden topics to capture latent semantics in text documents. However, it assumes that all the documents are represented by the same topics, resulting in the "forced topic" problem. To solve this problem, we developed a group latent Dirichlet allocation (GLDA). GLDA uses two kinds of topics: local topics and global topics. The highly related local topics are organized into groups to describe the local semantics, whereas the global topics are shared by all the documents to describe the background semantics. GLDA uses variational inference algorithms for both offline and online data. We evaluated the proposed model for topic modeling and document clustering. Our experimental results indicated that GLDA can achieve a competitive performance when compared with state-of-the-art approaches.

Text Categorization Comparison between Simple BPNN and Combinatorial Method of LSI and BPNN

paper proposed a text categorization comparison between simple BPNN and Combinatorial method of LSI and BPNN. In the traditional error back propagation network, the weight adjustment process gets block in a local minima and also the training speed of such network is very slow which leads to reduced performance and reduced efficiency of the network. Also the Learning time of overall network is very high.Hence, to improve the categorization accuracy, a new combinatorial method of LSI (latent semantic Indexing) and BPNN (back propagation neural network) is proposed. The latent semantics demonstration is an accurate data structure in low-dimensional space in which documents, terms and queries are rooted and also compared. Singular value decomposition (SVD) technique is used in Latent semantic Analysis in which large term-document matrix is decomposed into a set of k orthogonal factors by which the original textual data is changed to a smaller semantic space. New document vectors are found in reduced k-dimensional space. Also new coordinates of the queries are found. Here we implement combinatorial method of LSI and BPNN based technique for the classification of 20Newsgroup dataset which include categories of Sports, CS, and Medicine. The proposed technique implemented is compared with the existing BPNN technique. . Hence, this new method greatly reduces the dimension and better classification results can be achieved.

Application of a staged learning-based resource allocation network to automatic text categorization

In this paper, we propose a novel learning classifier which utilizes a staged learning-based resource allocation network (SLRAN) for text categorization. In the light of its learning progress, SLRAN is divided into a preliminary learning phase and a refined learning phase. In the former phase, to reduce the sensitivity corresponding to input data an agglomerate hierarchical k-means method is utilized to create the initial structure of hidden layer. Subsequently, a novelty criterion is put forward to dynamically regulate the hidden layer centers. In the latter phase a least square method is used to enhance the convergence rate of network and further improve its ability for classification. Such staged learning-based approach builds a compact structure which decreases the computational complexity of network and boosts its learning capability. In order to implement SLRAN to text categorization, we utilize a semantic similarity approach which reduces the input scales of neural network and reveals the latent semantics between text features. The benchmark Reuter and 20-newsgroup datasets are tested in our experiments and the extensive experimental results reveal that the dynamic learning process of SLRAN improves its classifying performance in comparison with conventional classifiers, e.g. RAN, BP, RBF neural networks and SVM.

Higher-Order Smoothing: A Novel Semantic Smoothing Method for Text Classification

It is known that latent semantic indexing (LSI) takes advantage of implicit higher-order (or latent) structure in the association of terms and documents. Higher-order relations in LSI capture “latent semantics”. These findings have inspired a novel Bayesian framework for classification named Higher-Order Naive Bayes (HONB), which was introduced previously, that can explicitly make use of these higher-order relations. In this paper, we present a novel semantic smoothing method named Higher-Order Smoothing (HOS) for the Naive Bayes algorithm. HOS is built on a similar graph based data representation of the HONB which allows semantics in higher-order paths to be exploited. We take the concept one step further in HOS and exploit the relationships between instances of different classes. As a result, we move beyond not only instance boundaries, but also class boundaries to exploit the latent information in higher-order paths. This approach improves the parameter estimation when dealing with insufficient labeled data. Results of our extensive experiments demonstrate the value of HOS on several benchmark datasets.

Taking advantage of improved resource allocating network and latent semantic feature selection approach for automated text categorization

In this study we propose an improved learning algorithm based on resource allocating network (RAN) for text categorization. RAN is a promising neural network of single hidden layer structure based on radial basis function. We firstly use the means clustering-based method to determine the initial centers in the hidden layer. Such method can effectively overcome the limitation of local-optimal of clustering algorithms. Subsequently, in order to improve the novelty criteria of RAN, we propose a root mean square (RMS) sliding window method which can reduce the underlying influence of undesirable noise data. Through the further research on the learning process of RAN, we divide the learning process of RAN into a preliminary study phase and a subsequent study phase. The former phase initializes the preliminary structure of RAN and decreases the complexity of network, while the latter phase refines its learning ability and improves the classification accuracy. Such a compact network structure decreases the computational complexity and maintains the higher convergence rate. Moreover, a latent semantic feature selection method is utilized to organize documents. This method reduces the input scale of network, and reveals the latent semantics between features. Extensive experiments are conducted on two benchmark datasets, and the results demonstrate the superiority of our algorithm in comparison with state of the art text categorization algorithms.

Contextual modeling for logical labeling of PDF documents

The widely-used Portable Document Format (PDF) documents are known to be layout-oriented and not suitable for mobile applications. In this paper, a Conditional Random Fields (CRF) based model is proposed to learn latent semantics of PDF page content. Local and contextual observations constructed from PDF attributes are incorporated to facilitate the determination of semantic roles. The observations are carefully designed to work even in different styles of documents. A local classifier is first used to generate posterior probabilities. The local estimate is then fed to the CRF model for joint classification. The experimental results evidently approve the positive effects of contextual information in logical labeling. Our work has revealed the potential usability of existing born-digital fixed-layout documents for mobile applications.

Latent Semantic Analysis for Multimodal User Input With Speech and Gestures

This paper describes our work in semantic interpretation of a “multimodal language” with speech and gestures using latent semantic analysis (LSA). Our aim is to infer the domain-specific informational goal of multimodal inputs. The informational goal is characterized by lexical terms used in the spoken modality, partial semantics of gestures in the pen modality, as well as term co-occurrence patterns across modalities, leading to “multimodal terms.” We designed and collected a multimodal corpus of navigational inquiries. We also obtained perfect (i.e. manual) and imperfect (i.e. automatic via recognition) transcriptions for these. We automatically align parsed spoken locative references (SLRs) with their corresponding pen gesture(s) using the Viterbi alignment, according to their numeric and location type features. Then, we characterize each cross-modal integration pattern as a 3-tuple multimodal term with SLR, pen gesture type and their temporal relationship. We propose to use latent semantic analysis (LSA) to derive the latent semantics from manual (i.e. perfect) and automatic (i.e. imperfect) transcriptions of the collected multimodal inputs. In order to achieve this, both multimodal and lexical terms are used to compose an inquiry-term matrix, which is then factorized using singular value decomposition (SVD) to derive the latent semantics automatically. Informational goal inference based on the latent semantics shows that the informational goal inference accuracy of a disjoint test set is 99% and 84% when a perfect and imperfect projection model is used respectively, which performs significantly better than (at least 9.9% absolute) the baseline performance using vector-space model (VSM).

Biomedical Literature Exploration through Latent Semantics

The fast increasing amount of articles published in the biomedical field is creating difficulties in the way this wealth of information can be efficiently exploited by researchers. As a way of overcoming these limitations and potentiating a more efficient use of the literature, we propose an approach for structuring the results of a literature search based on the latent semantic information extracted from a corpus. Moreover, we show how the results of the Latent Semantic Analysis method can be adapted so as to evidence differences between results of different searches. We also propose different visualization techniques that can be applied to explore these results. Used in combination, these techniques could empower users with tools for literature guided knowledge exploration and discovery.

Structure preserving non-negative matrix factorization for dimensionality reduction

The problem of dimensionality reduction is to map data from high dimensional spaces to low dimensional spaces. In the process of dimensionality reduction, the data structure, which is helpful to discover the latent semantics and simultaneously respect the intrinsic geometric structure, should be preserved. In this paper, to discover a low-dimensional embedding space with the nature of structure preservation and basis compactness, we propose a novel dimensionality reduction algorithm, called Structure Preserving Non-negative Matrix Factorization (SPNMF). In SPNMF, three kinds of constraints, namely local affinity, distant repulsion, and embedding basis redundancy elimination, are incorporated into the NMF framework. SPNMF is formulated as an optimization problem and solved by an effective iterative multiplicative update algorithm. The convergence of the proposed update solutions is proved. Extensive experiments on both synthetic data and six real world data sets demonstrate the encouraging performance of the proposed algorithm in comparison to the state-of-the-art algorithms, especially some related works based on NMF. Moreover, the convergence of the proposed updating rules is experimentally validated.

Clustering facilitated web services discovery model based on supervised term weighting and adaptive metric learning

With the explosive growth of web services, the research on how to rapidly find the desired services becomes increasingly important and challenging. In this paper, we focus on non-semantic web services discovery and present an efficient clustering facilitated web services discovery model (CFWSFinder). Compared with the existing models, CFWSFinder has several characteristics. First, in services representing process, CFWSFinder imports WordNet and latent semantics index to represent non-semantic web services as the low-dimensional compact semantic feature vectors; Second, in services clustering process, CFWSFinder employs a modified Kernel batch self-organising map (KBSOM) neural network to minimise the services discovery duration; Third and most importantly, in services matching process, by using the category label information achieved from services clustering process, CFWSFinder can adopt supervised term weighting scheme and adaptive metric learning method to ameliorate the services discovery precision. Finally, experimental results performed on the real-world web services collection demonstrate the feasibility of the CFWSFinder.

Exposing, formalizing and reasoning over the latent semantics of tags in multimodal data sources

This paper describes our solution to the problem of inducing ontological information from metadata provided informally. The metadata is in the form of linguistic tags attached to items in an on-line domain. We formulate four hypotheses about the structure implicit in a set of tags and evaluate them using data from public tag sets. The results confirm three of the four hypotheses and show that it is feasible for ontological information --specifically subclass relationships --to be made explicit and hence available for inferencing.

Latent semantic learning with structured sparse representation for human action recognition

This paper proposes a novel latent semantic learning method for extracting high-level latent semantics from a large vocabulary of abundant mid-level features (i.e. visual keywords) with structured sparse representation, which can help to bridge the semantic gap in the challenging task of human action recognition. To discover the manifold structure of mid-level features, we develop a graph-based spectral embedding approach to latent semantic learning, with the graph over mid-level features being constructed using sparse representation. Moreover, we define new L1-norm hypergraph regularization to induce extra structured sparsity into sparse representation for graph construction. Due to the nice properties (sparsity and noise-robustness) of such structured sparse representation, our graph construction can capture dominant and robust relationships among mid-level features, which are crucial for the success of latent semantic learning in action recognition. Unlike the traditional latent semantic analysis based on topic models, our latent semantic learning method can explore the manifold structure of mid-level features in both graph construction and spectral embedding, which results in compact but discriminative high-level features. The experimental results on the commonly used KTH action dataset and unconstrained YouTube action dataset show the promising performance of our method.