External Knowledge Bases Research Articles

Keyphrase extraction from single textual documents based on semantically defined background knowledge and co-occurrence graphs

The keyphrase extraction task is a fundamental and challenging task designed to extract a set of keyphrases from textual documents. Keyphrases are essential to assist publishers in indexing documents and readers in identifying the most relevant ones. They are short phrases composed of one or more terms used to represent a textual document and its main topics. In this article, we extend our research on C-Rank, which is an unsupervised approach that automatically extracts keyphrases from single documents. C-Rank uses concept-linking to link concepts in common between single documents and an external background knowledge base. We advance our study over C-Rank by evaluating it using different concept-linking approaches - Babelfy and DBPedia Spotlight. We evaluated C-Rank on data sets composed of academic articles, academic abstracts, and news articles. Our findings indicate that C-Rank achieves state-of-the-art results extracting keyphrases from scientific documents by experimentally comparing it to existing unsupervised approaches.

Keyphrase extraction from single textual documents based on semantically defined background knowledge and co-occurrence graphs

The keyphrase extraction task is a fundamental and challenging task designed to extract a set of keyphrases from textual documents. Keyphrases are essential to assist publishers in indexing documents and readers in identifying the most relevant ones. They are short phrases composed of one or more terms used to represent a textual document and its main topics. In this article, we extend our research on C-Rank, which is an unsupervised approach that automatically extracts keyphrases from single documents. C-Rank uses concept-linking to link concepts in common between single documents and an external background knowledge base. We advance our study over C-Rank by evaluating it using different concept-linking approaches - Babelfy and DBPedia Spotlight. We evaluated C-Rank on data sets composed of academic articles, academic abstracts, and news articles. Our findings indicate that C-Rank achieves state-of-the-art results extracting keyphrases from scientific documents by experimentally comparing it to existing unsupervised approaches.

POI Classification Method Based on Feature Extension and Deep Learning

The automatic classification of point of interest (POI) function types based on POI name texts and intelligent computing can provide convenience in travel recommendations, map information queries, urban function divisions, and other services. However, POI name texts belong to short texts, which few characters and sparse features. Therefore, it is difficult to guarantee the feature learning ability and classification effect of the model when distinguishing the POI function types. This paper proposes a POI classification method based on feature extension and deep learning to establish a short-text classification model. We utilize an Internet search engine as an external knowledge base to introduce real-time, large-scale text feature information to the original POI text to solve the limitation of sparse POI name text features. The input text information is represented by the attention calculation matrix used to reduce the noise information of the extended text and the word-embedding matrix of the original text. We utilize a convolutional neural network with excellent local feature extraction ability to establish the classification model. Experimental results on a real-world dataset (obtained from Baidu) show the excellent performance of our model in POI classification tasks compared with other baseline models.

Find truth in the hands of the few: acquiring specific knowledge with crowdsourcing

Crowdsourcing has been a helpful mechanism to leverage human intelligence to acquire useful knowledge. However, when we aggregate the crowd knowledge based on the currently developed voting algorithms, it often results in common knowledge that may not be expected. In this paper, we consider the problem of collecting specific knowledge via crowdsourcing. With the help of using external knowledge base such as WordNet, we incorporate the semantic relations between the alternative answers into a probabilistic model to determine which answer is more specific. We formulate the probabilistic model considering both worker’s ability and task’s difficulty from the basic assumption, and solve it by the expectation-maximization (EM) algorithm. To increase algorithm compatibility, we also refine our method into semi-supervised one. Experimental results show that our approach is robust with hyper-parameters and achieves better improvement than majority voting and other algorithms when more specific answers are expected, especially for sparse data.

Internal and External Sources of Knowledge in Manufacturing and Service Enterprises. A Comparative Analysis of European Union Countries

Along with the development of economies based on knowledge, the importance of knowledge input in production processes has been increasing. Enterprises may acquire knowledge input by developing their internal knowledge base and/or purchasing knowledge from external entities. Their internal knowledge base may be developed mainly by employing highly qualified specialists and their own research. The aim of the paper is to examine the importance of all these knowledge sources in manufacturing and services enterprises, as well as to compare their changing role with productivity performance in EU countries. It is based on data from the World Input-Output Database, Eurostat, OECD and EU KLEMS. Thanks to the availability of relevant data, the analysed period covers the years 1995–2018. The study demonstrates that knowledge base, developed through both internal and external sources, played a significantly more important role in the EU-15 than in the EU-12, with a tendency to decrease these disparities (most visible with respect to KIBS input). The growing importance of an external knowledge base was more visible and stable in the EU-12 countries. R&D expenditures were an exception. The recent financial crisis heavily affected only external R&D expenditures.

A Mathematical Model for Universal Semantics.

We characterize the meaning of words with language-independent numerical fingerprints, through a mathematical analysis of recurring patterns in texts. Approximating texts by Markov processes on a long-range time scale, we are able to extract topics, discover synonyms, and sketch semantic fields from a particular document of moderate length, without consulting external knowledge-base or thesaurus. Our Markov semantic model allows us to represent each topical concept by a low-dimensional vector, interpretable as algebraic invariants in succinct statistical operations on the document, targeting local environments of individual words. These language-independent semantic representations enable a robot reader to both understand short texts in a given language (automated question-answering) and match medium-length texts across different languages (automated word translation). Our semantic fingerprints quantify local meaning of words in 14 representative languages across five major language families, suggesting a universal and cost-effective mechanism by which human languages are processed at the semantic level. Our protocols and source codes are publicly available on https://github.com/yajun-zhou/linguae-naturalis-principia-mathematica.

Gains or pains? Investigating effects of R&D collaboration intensity and technological diversification on new product development

ABSTRACT Research and development (R&D) collaboration plays a vital role in the development of new products. As both external knowledge sourcing and internal knowledge base contribute to new product development (NPD), this study examines how a firm’s R&D collaboration intensity and internal knowledge base properties (i.e. related and unrelated technological diversification) affect its NPD performance. To test the proposed model, we manually collect multi-source data such as patents and new product announcements in the Chinese automobile industry. The empirical analysis reveals that the relationship between R&D collaboration intensity and NPD performance is inverted-U-shaped. Moreover, an increase in a firm’s related technological diversification steepens the inverted-U relationship, while an increase in unrelated technological diversification flattens the curve. This study contributes to the literature on R&D collaboration and NPD by elaborating the contingent roles of related and unrelated technological diversification. As NPD activities are usually knowledge-oriented, firms should commit more seriously to their internal knowledge-based properties when making the decision of R&D collaboration solutions.

Knowledge-Enhanced Graph Neural Networks for Sequential Recommendation

With the rapid increase in the popularity of big data and internet technology, sequential recommendation has become an important method to help people find items they are potentially interested in. Traditional recommendation methods use only recurrent neural networks (RNNs) to process sequential data. Although effective, the results may be unable to capture both the semantic-based preference and the complex transitions between items adequately. In this paper, we model separated session sequences into session graphs and capture complex transitions using graph neural networks (GNNs). We further link items in interaction sequences with existing external knowledge base (KB) entities and integrate the GNN-based recommender with key-value memory networks (KV-MNs) to incorporate KB knowledge. Specifically, we set a key matrix to many relation embeddings that learned from KB, corresponding to many entity attributes, and set up a set of value matrices storing the semantic-based preferences of different users for the corresponding attribute. By using a hybrid of a GNN and KV-MN, each session is represented as the combination of the current interest (i.e., sequential preference) and the global preference (i.e., semantic-based preference) of that session. Extensive experiments on three public real-world datasets show that our method performs better than baseline algorithms consistently.

Heterogeneous classifier ensemble for sentiment analysis of Bengali and Hindi tweets

Sentiment analysis is an essential step for analysing social media texts such as tweets and other posts on the various micro-blogging sites. The basic step of sentiment analysis is sentiment polarity detection, which identifies whether an input piece of social media text is positive, negative or neutral. In this paper, we present an approach that combines heterogeneous classifiers in an ensemble for sentiment polarity detection in Bengali and Hindi tweets. Our proposed method constructs an ensemble of three different base classifiers where the feature set for each base classifier is different from each other. We have also incorporated an external knowledge base called sentiment lexicon to augment tweet words with sentiment polarity information retrieved from the sentiment lexicon. Experimental results show the effectiveness of our proposed heterogeneous ensemble model for sentiment polarity detection for both Bengali and Hindi languages. It has been shown that our system outperforms other existing Bengali and Hindi sentiment classification systems to which it is compared.

Incorporation of knowledge through acquisition in the pharmaceutical industry

This article focuses on the incorporation of knowledge conducted by large pharmaceutical corporations through the acquisition of small enterprises. The objective was to evidence to which degree the incorporation of knowledge is performed by a sample of 8 large pharmaceutical companies. In order to do this, we rely on qualitative methods, based on a sample of eight companies and 54 small enterprises they acquired in the period from 2005 to 2012. From the sample we compiled the patents granted to the small firms within the USPTO. The analysis conducted develops two central actions. The first is theinventor’s usage, defined as the small enterprises inventors, which when incorporated by large pharmaceutical corporations started to develop patents through the acquiring larger company. The second is the incorporation of research trajectories, which means how inventors cite their past work as they are incorporated into the larger companies. This article concludes that the incorporation of inventors is a relevant strategy among the large companies studied so that these companies can incorporate external knowledge bases from the acquired smaller enterprises; however, just a few inventors are incorporated, which shows that just a small group of people conducts innovative research for large companies. This article contribution was to categorically evidence the incorporation of knowledge through new metrics, being them: inventor’s usage and research trajectory incorporation.

A New Text Sentiment Analysis Method Based on Chinese Morphological Features and HowNet

Traditional deep learning methods have two problems when using vectorized words as input. One is that they only consider the overall semantic information of the vocabulary, but ignore the morphological features of the Chinese vocabulary and the prior knowledge of the Chinese external knowledge base. Second, the word vector corresponding to each word will be limited to a single word vector training model. Aiming at these problems, we propose a double-channel convolutional neural network model based on Chinese morphological features and HowNet. First, the cw2vec model and the SAT model (Sememe Attention over Target Model) are used to train the word vectors. Second, the two different word vectors are used as the input of the two channels of the model. Finally, the convolutional neural network are used to extract the characteristics of the two channels to complete the sentiment analysis task. The comparative experimental results on the two data sets show that the proposed model achieves significantly better classification performance than traditional sentiment analysis methods.

Improving the robustness of machine reading comprehension model with hierarchical knowledge and auxiliary unanswerability prediction

Machine Reading Comprehension (MRC) aims to understand a passage and answer a series of related questions. With the development of deep learning and the release of large-scale MRC datasets, many end-to-end MRC neural networks have achieved remarkable success. However, these models are fragile and lack of robustness when there are some imperceptible adversarial perturbations in the input. In this paper, we propose an MRC model which has two main components to improve the robustness. On the one hand, we enhance the representation of the model by leveraging hierarchical knowledge from external knowledge bases. On the other hand, we introduce an auxiliary unanswerability prediction module and perform supervised multi-task learning along with a span prediction task. Experimental results on benchmark datasets show that our model can achieve consistent improvement compared with other strong baselines.

Attentive interaction-driven entity resolution over multi-source web information

The task of multi-source web entity resolution (MSWER) aims to automatically discover entity references from multiple web sources that refer to the same real-world entity, which plays an important role in tasks such as question answering and recommendations. However, existing approaches typically suffer from three major limitations: (1) they usually treat the MSWER as an information retrieval task and focus on learning the similarity between entity references based on the associated features extracted from multiple sources; (2) they ignore the valuable implicit interactions between the associated features of different entities that cannot be directly captured based on the given data without any external knowledge; (3) they didn’t consider the redundant and noisy interactions between features. To overcome these limitations, this paper presents a novel attentive interaction-driven entity resolution model (AIDER). Our theme is to capture both the explicit and implicit interactions of features associated with entity references in the form of paths, and further develop an end-to-end entity resolution model for inferring the equivalent entity references. Accordingly, an external knowledge base is leveraged to construct paths for implicit interactions, and a well-designed attention mechanism is further employed to measure the importance of each path-based interaction, which focuses on useful interactions and neglects those redundant and noisy ones. Experimental results on three real-world datasets demonstrate that AIDER outperforms the state-of-the-art approaches.

Knowledge-Graph Augmented Word Representations for Named Entity Recognition

By modeling the context information, ELMo and BERT have successfully improved the state-of-the-art of word representation, and demonstrated their effectiveness on the Named Entity Recognition task. In this paper, in addition to such context modeling, we propose to encode the prior knowledge of entities from an external knowledge base into the representation, and introduce a Knowledge-Graph Augmented Word Representation or KAWR for named entity recognition. Basically, KAWR provides a kind of knowledge-aware representation for words by 1) encoding entity information from a pre-trained KG embedding model with a new recurrent unit (GERU), and 2) strengthening context modeling from knowledge wise by providing a relation attention scheme based on the entity relations defined in KG. We demonstrate that KAWR, as an augmented version of the existing linguistic word representations, promotes F1 scores on 5 datasets in various domains by +0.46∼+2.07. Better generalization is also observed for KAWR on new entities that cannot be found in the training sets.

Efficient Weighted Semantic Score Based on the Huffman Coding Algorithm and Knowledge Bases for Word Sequences Embedding

Learning text representation is forming a core for numerous natural language processing applications. Word embedding is a type of text representation that allows words with similar meaning to have similar representation. Word embedding techniques categorize semantic similarities between linguistic items based on their distributional properties in large samples of text data. Although these techniques are very efficient, handling semantic and pragmatics ambiguity with high accuracy is still a challenging research task. In this article, we propose a new feature as a semantic score which handles ambiguities between words. We use external knowledge bases and the Huffman Coding algorithm to compute this score that depicts the semantic relatedness between all fragments composing a given text. We combine this feature with word embedding methods to improve text representation. We evaluate our method on a hashtag recommendation system in Twitter where text is noisy and short. The experimental results demonstrate that, compared with state-of-the-art algorithms, our method achieves good results.

Bag-of-Concepts representation for document classification based on automatic knowledge acquisition from probabilistic knowledge base

Text representation, a crucial step for text mining and natural language processing, concerns about transforming unstructured textual data into structured numerical vectors to support various machine learning and data mining algorithms. For document classification, one classical and commonly adopted text representation method is Bag-of-Words (BoW) model. BoW represents document as a fixed-length vector of terms, where each term dimension is a numerical value such as term frequency or tf-idf weight. However, BoW simply looks at surface form of words. It ignores the semantic, conceptual and contextual information of texts, and also suffers from high dimensionality and sparsity issues. To address the aforementioned issues, we propose a novel document representation scheme called Bag-of-Concepts (BoC), which automatically acquires useful conceptual knowledge from external knowledge base, then conceptualizes words and phrases in the document into higher level semantics (i.e. concepts) in a probabilistic manner, and eventually represents a document as a distributed vector in the learned concept space. By utilizing background knowledge from knowledge base, BoC representation is able to provide more semantic and conceptual information of texts, as well as better interpretability for human understanding. We also propose Bag-of-Concept-Clusters (BoCCl) model which clusters semantically similar concepts together and performs entity sense disambiguation to further improve BoC representation. In addition, we combine BoCCl and BoW representations using an attention mechanism to effectively utilize both concept-level and word-level information and achieve optimal performance for document classification.

Cross-Media Semantic Correlation Learning Based on Deep Hash Network and Semantic Expansion for Social Network Cross-Media Search.

Cross-media search from large-scale social network big data has become increasingly valuable in our daily life because it can support querying different data modalities. Deep hash networks have shown high potential in achieving efficient and effective cross-media search performance. However, due to the fact that social network data often exhibit text sparsity, diversity, and noise characteristics, the search performance of existing methods often degrades when dealing with this data. In order to address this problem, this article proposes a novel end-to-end cross-media semantic correlation learning model based on a deep hash network and semantic expansion for social network cross-media search (DHNS). The approach combines deep network feature learning and hash-code quantization learning for multimodal data into a unified optimization architecture, which successfully preserves both intramedia similarity and intermedia correlation, by minimizing both cross-media correlation loss and binary hash quantization loss. In addition, our approach realizes semantic relationship expansion by constructing the image-word relation graph and mining the potential semantic relationship between images and words, and obtaining the semantic embedding based on both internal graph deep walk and an external knowledge base. Experimental results demonstrate that DHNS yields better cross-media search performance on standard benchmarks.

Knowledge-Based Topic Model for Multi-Modal Social Event Analysis

With the accumulation of data on the Internet and progress in representation learning techniques, knowledge priors learned from a large-scale knowledge base has been increasingly used in probabilistic topic models. However, it is challenging to learn interpretable topics and a discriminative event representation based on multi-modal information. To address these issues, we propose a knowledge priors- and max-margin-based topic model for multi-modal social event analysis, called the KGE-MMSLDA, in which feature representation and knowledge priors are jointly learned. Our model has three main advantages over current methods: (1) It integrates additional knowledge from external knowledge base into a unified topic model in which the max-margin classifier, and multi-modal information are exploited to increase the number of event descriptions obtained. (2) We mined knowledge priors from over 74,000 web documents. Multi-modal data with these knowledge priors are then incorporated into the topic model to increase the number of coherent topics learned. (3) A large-scale multi-modal dataset (containing 10 events, where each event contained approximately 7,000 Flickr pages) was collected and has been released publicly for event topic mining and classification research. In comparative experiments, the proposed method outperformed state-of-the-art models on topic coherence, and obtained a classification accuracy of 85.1%.

Enhancing the Quality of Image Tagging Using a Visio-Textual Knowledge Base

Auto-tagging of images is important for image understanding and for tag-based applications viz. image retrieval, visual question-answering, image captioning, etc. Although existing tagging methods incorporate both visual and textual information to assign/refine tags, they lag in tag-image relevance, completeness, and preciseness, thereby resulting in the unsatisfactory performance of tag-based applications. In order to bridge this gap, we propose a novel framework for tag assignment using knowledge embedding (TAKE) from a proposed external knowledge base, considering properties such as Rarity, Newness, Generality, and Naturalness ( RNGN properties). These properties help in providing a rich semantic representation to images. Existing knowledge bases provide multiple types of relations extracted through only one modality, either text or visual, which is not effective in image related applications. We construct a simple yet effective Visio-Textual Knowledge Base (VTKB) with only four relations using reliable resources such as Wikipedia, thesauruses, dictionaries, etc. Our large scale experiments demonstrate that the proposed combination of TAKE and VTKB assigns a large number of high quality tags in comparison to the ConceptNet and ImageNet knowledge bases when used in conjunction with TAKE. Also, the effectiveness of knowledge embedding through VTKB is evaluated for image tagging and tag-based image retrieval (TBIR).

Mining Entity Synonyms with Efficient Neural Set Generation

Mining entity synonym sets (i.e., sets of terms referring to the same entity) is an important task for many entity-leveraging applications. Previous work either rank terms based on their similarity to a given query term, or treats the problem as a two-phase task (i.e., detecting synonymy pairs, followed by organizing these pairs into synonym sets). However, these approaches fail to model the holistic semantics of a set and suffer from the error propagation issue. Here we propose a new framework, named SynSetMine, that efficiently generates entity synonym sets from a given vocabulary, using example sets from external knowledge bases as distant supervision. SynSetMine consists of two novel modules: (1) a set-instance classifier that jointly learns how to represent a permutation invariant synonym set and whether to include a new instance (i.e., a term) into the set, and (2) a set generation algorithm that enumerates the vocabulary only once and applies the learned set-instance classifier to detect all entity synonym sets in it. Experiments on three real datasets from different domains demonstrate both effectiveness and efficiency of SynSetMine for mining entity synonym sets.