Tree Kernel Research Articles

Matching parse thickets for open domain question answering

Traditional parse trees are combined together and enriched with anaphora and rhetoric information to form a unified representation for a paragraph of text. We refer to these representations as parse thickets. They are introduced to support answering complex questions, which include multiple sentences, to tackle as many constraints expressed in this question as possible. The question answering system is designed so that an initial set of answers, which is obtained by a TF*IDF or other keyword search model, is re-ranked. Passage re-ranking is performed using matching of the parse thickets of answers with the parse thicket of the question. To do that, a graph representation and matching technique for parse structures for paragraphs of text have been developed. We define the operation of generalization of two parse thickets as a measure of the distance between paragraphs of text to be the maximal common sub-graph of these parse thickets. A partial case of parse thickets, a rhetoric map of an answer, allows leveraging discourse for relevance in a rule-based manner.Passage re-ranking improvement via parse thickets is evaluated in a variety of search domains with long questions. Using parse thickets improves search accuracy compared with the bag-of words, the pairwise matching of parse trees for sentences, and the tree kernel approaches. As a baseline, we use a web search engine API, which provides much more accurate search results than the majority of search benchmarks, such as TREC. A comparative analysis of the impact of various sources of discourse information on the search accuracy is conducted. An open source plug-in for SOLR is developed so that the proposed technology can be easily integrated with industrial search engines.

The content and emission factors of heavy metals in biomass used for energy purposes in the context of the requirements of international standards

The aim of this work was to study the content of seven heavy metal factors within different biomasses. The potential environmental effects associated with the release of these elements into the atmosphere were also studied by determining the emission factors for each of biomass. 31 samples of biomass which were classified into four groups, i.e.: wood biomass, herbaceous biomass, fruit biomass and aquatic biomass and analyzed with use of ICP-OES spectrometer. Performed results were analyzed in the context of the requirements of international standards, described in the EN ISO 17225:2014. Higher heavy metal contents than the limits described in the EN ISO 17225:2014 standard were detected within some of the studied biomass samples. Importantly, this is also related to the products available on the market, i.e., wood pellets or wood chips. Among the detected trace elements, the largest limit exceedance was for lead (wood chips from a rubber tree, pomace oil and palm kernel shells) and arsenic (Sea Balls). The highest emission factors were determined for the herbaceous, fruit and aquatic biomass. However, the wood chips studied were characterized by relatively high emission factors for each heavy metal. A comparison of the emission factors calculated for the studied biomass samples and for coal revealed that the total emission of heavy metals during the combustion of wood pellets and wood chips was lower than that calculated for hard coal.

An ensemble method for extracting adverse drug events from social media

Because adverse drug events (ADEs) are a serious health problem and a leading cause of death, it is of vital importance to identify them correctly and in a timely manner. With the development of Web 2.0, social media has become a large data source for information on ADEs. The objective of this study is to develop a relation extraction system that uses natural language processing techniques to effectively distinguish between ADEs and non-ADEs in informal text on social media. We develop a feature-based approach that utilizes various lexical, syntactic, and semantic features. Information-gain-based feature selection is performed to address high-dimensional features. Then, we evaluate the effectiveness of four well-known kernel-based approaches (i.e., subset tree kernel, tree kernel, shortest dependency path kernel, and all-paths graph kernel) and several ensembles that are generated by adopting different combination methods (i.e., majority voting, weighted averaging, and stacked generalization). All of the approaches are tested using three data sets: two health-related discussion forums and one general social networking site (i.e., Twitter). When investigating the contribution of each feature subset, the feature-based approach attains the best area under the receiver operating characteristics curve (AUC) values, which are 78.6%, 72.2%, and 79.2% on the three data sets. When individual methods are used, we attain the best AUC values of 82.1%, 73.2%, and 77.0% using the subset tree kernel, shortest dependency path kernel, and feature-based approach on the three data sets, respectively. When using classifier ensembles, we achieve the best AUC values of 84.5%, 77.3%, and 84.5% on the three data sets, outperforming the baselines. Our experimental results indicate that ADE extraction from social media can benefit from feature selection. With respect to the effectiveness of different feature subsets, lexical features and semantic features can enhance the ADE extraction capability. Kernel-based approaches, which can stay away from the feature sparsity issue, are qualified to address the ADE extraction problem. Combining different individual classifiers using suitable combination methods can further enhance the ADE extraction effectiveness.

Ordered Decompositional DAG kernels enhancements

In this paper, we show how the Ordered Decomposition DAGs (ODD) kernel framework, a framework that allows the definition of graph kernels from tree kernels, allows to easily define new state-of-the-art graph kernels. Here we consider a fast graph kernel based on the Subtree kernel (ST), and we propose various enhancements to increase its expressiveness. The proposed DAG kernel has the same worst-case complexity as the one based on ST, but an improved expressivity due to an augmented set of features. Moreover, we propose a novel weighting scheme for the features, which can be applied to other kernels of the ODD framework. These improvements allow the proposed kernels to improve on the classification performances of the ST-based kernel for several real-world datasets, reaching state-of-the-art performances.

PIPE: a protein-protein interaction passage extraction module for BioCreative challenge.

Identifying the interactions between proteins mentioned in biomedical literatures is one of the frequently discussed topics of text mining in the life science field. In this article, we propose PIPE, an interaction pattern generation module used in the Collaborative Biocurator Assistant Task at BioCreative V (http://www.biocreative.org/) to capture frequent protein-protein interaction (PPI) patterns within text. We also present an interaction pattern tree (IPT) kernel method that integrates the PPI patterns with convolution tree kernel (CTK) to extract PPIs. Methods were evaluated on LLL, IEPA, HPRD50, AIMed and BioInfer corpora using cross-validation, cross-learning and cross-corpus evaluation. Empirical evaluations demonstrate that our method is effective and outperforms several well-known PPI extraction methods.Database URL:

Support Vector Machine with Ensemble Tree Kernel for Relation Extraction.

Relation extraction is one of the important research topics in the field of information extraction research. To solve the problem of semantic variation in traditional semisupervised relation extraction algorithm, this paper proposes a novel semisupervised relation extraction algorithm based on ensemble learning (LXRE). The new algorithm mainly uses two kinds of support vector machine classifiers based on tree kernel for integration and integrates the strategy of constrained extension seed set. The new algorithm can weaken the inaccuracy of relation extraction, which is caused by the phenomenon of semantic variation. The numerical experimental research based on two benchmark data sets (PropBank and AIMed) shows that the LXRE algorithm proposed in the paper is superior to other two common relation extraction methods in four evaluation indexes (Precision, Recall, F-measure, and Accuracy). It indicates that the new algorithm has good relation extraction ability compared with others.

Distributed Smoothed Tree Kernel

In this paper we explore the possibility to merge the world of Compositional Distributional Semantic Models (CDSM) with Tree Kernels (TK). In particular, we will introduce a specific tree kernel (smoothed tree kernel, or STK) and then show that is possibile to approximate such kernel with the dot product of two vectors obtained compositionally from the sentences, creating in such a way a new CDSM.

Event causality extraction based on connectives analysis

Causality is an important type of relation which is crucial in numerous tasks, such as predicting future events, generating scenario, question answering, textual entailment and discourse comprehension. Therefore, causality extraction is a fundamental task in text mining. Many efforts have been dedicated to extracting causality from texts utilizing patterns, constraints and machine learning techniques. This paper presents a new Restricted Hidden Naive Bayes model to extract causality from texts. Besides some commonly used features, such as contextual features, syntactic features, position features, we also utilize a new category feature of causal connectives. This new feature is obtained from the tree kernel similarity of sentences containing connectives. In previous studies, the features have been usually assumed to be independent, which is not the case in reality. The advantage of our model lies in its ability to cope with partial interactions among features so as to avoid over-fitting problem on Hidden Naive Bayes model, especially the interaction between the connective category and the syntactic structure of sentences. Evaluation on a public dataset shows that our method goes beyond all the baselines.

A theory of subtree matching and tree kernels based on the edit distance concept

Edit distances provide us with an established method to capture structural features of data, and a distance between data objects represents their dissimilarity. In contrast, kernels form a category of similarity functions, and a positive definite kernel enables us to leverage abundant techniques of multivariate analysis. This paper aims to fill the gap between distances and kernels. In the literature, we have several formulas that convert a negative definite distance function into a positive definite kernel. Edit distance functions, however, are not necessarily negative definite, and our first contribution is to introduce an alternative method to derive positive definite kernels from edit distance functions that are not necessarily negative definite. The method is equipped with an easy-to-check and strong sufficient condition for positive definiteness, and the condition turns out to be tightly related with the triangle inequality. In fact, to our knowledge, all of the edit distance functions in the literature that support the triangle inequality meet the condition for positive definiteness. Secondly, we apply this method to four well-known edit distance functions for trees to introduce four novel kernels and show that three of them are positive definite. Thirdly, we develop a theory of subtree matching to study these kernels. Our kernels count matchings between subtrees of the input trees with weights determined according to individual matchings. Although the number of such matchings is an exponential function of the size of the input trees (the number of vertices), our theory enables us to develop dynamic-programming-based algorithms, whose asymptotic computational complexities fall between a quadratic function and a cubic function of the size.

Phylodynamic Inference with Kernel ABC and Its Application to HIV Epidemiology.

The shapes of phylogenetic trees relating virus populations are determined by the adaptation of viruses within each host, and by the transmission of viruses among hosts. Phylodynamic inference attempts to reverse this flow of information, estimating parameters of these processes from the shape of a virus phylogeny reconstructed from a sample of genetic sequences from the epidemic. A key challenge to phylodynamic inference is quantifying the similarity between two trees in an efficient and comprehensive way. In this study, I demonstrate that a new distance measure, based on a subset tree kernel function from computational linguistics, confers a significant improvement over previous measures of tree shape for classifying trees generated under different epidemiological scenarios. Next, I incorporate this kernel-based distance measure into an approximate Bayesian computation (ABC) framework for phylodynamic inference. ABC bypasses the need for an analytical solution of model likelihood, as it only requires the ability to simulate data from the model. I validate this “kernel-ABC” method for phylodynamic inference by estimating parameters from data simulated under a simple epidemiological model. Results indicate that kernel-ABC attained greater accuracy for parameters associated with virus transmission than leading software on the same data sets. Finally, I apply the kernel-ABC framework to study a recent outbreak of a recombinant HIV subtype in China. Kernel-ABC provides a versatile framework for phylodynamic inference because it can fit a broader range of models than methods that rely on the computation of exact likelihoods.

The Research of Improved Spanning Tree Kernel Algorithm for Image Classification

The Research of Improved Spanning Tree Kernel Algorithm for Image Classification

Multi-lingual opinion mining on YouTube

In order to successfully apply opinion mining (OM) to the large amounts of user-generated content produced every day, we need robust models that can handle the noisy input well yet can easily be adapted to a new domain or language. We here focus on opinion mining for YouTube by (i) modeling classifiers that predict the type of a comment and its polarity, while distinguishing whether the polarity is directed towards the product or video; (ii) proposing a robust shallow syntactic structure (STRUCT) that adapts well when tested across domains; and (iii) evaluating the effectiveness on the proposed structure on two languages, English and Italian. We rely on tree kernels to automatically extract and learn features with better generalization power than traditionally used bag-of-word models. Our extensive empirical evaluation shows that (i) STRUCT outperforms the bag-of-words model both within the same domain (up to 2.6% and 3% of absolute improvement for Italian and English, respectively); (ii) it is particularly useful when tested across domains (up to more than 4% absolute improvement for both languages), especially when little training data is available (up to 10% absolute improvement) and (iii) the proposed structure is also effective in a lower-resource language scenario, where only less accurate linguistic processing tools are available.

A novel approach to wavelet selection and tree kernel construction for diagnosis of rolling element bearing fault

A novel methodology for early diagnosis of rolling element bearing fault is employed based on continuous wavelet transform (CWT) and support vector machine (SVM). CWT is especially suited for analyzing non-stationary signals in time–frequency domain where time information is retained as well as frequency content. To better approximate non-stationary vibration signals from rolling element bearing, a wavelet choice criterion is established to select an appropriate mother wavelet for feature extraction. The Shannon wavelet is picked out of several considered wavelets. The classification tree kernels (CTK) are constructed to address nonlinear classification of the characteristic samples derived from the wavelet coefficients. By using Fuzzy pruning strategy, a large variety of classification trees are generated. The trees with diverse structures can effectively explore intrinsic information among samples. Then, the tree kernel matrices can be acquired through ensemble statistical learning, which eventually reveal the similarity of samples objectively and stably. Under such architecture of kernel methods, a classification tree kernel based support vector machine (CTKSVM) is proposed to identify bearing fault. The performance of the methodology involving CWT and CTKSVM (CWT–CTKSVM) is evaluated by cross validation and independent test. The results show that the CWT–CTKSVM totally is superior to other SVM methods with common kernels. Therefore, it is a prospective technique for detection and identification of rolling element bearing fault.

Towards Topic-to-Question Generation

This paper is concerned with automatic generation of all possible questions from a topic of interest. Specifically, we consider that each topic is associated with a body of texts containing useful information about the topic. Then, questions are generated by exploiting the named entity information and the predicate argument structures of the sentences present in the body of texts. The importance of the generated questions is measured using Latent Dirichlet Allocation by identifying the subtopics (which are closely related to the original topic) in the given body of texts and applying the Extended String Subsequence Kernel to calculate their similarity with the questions. We also propose the use of syntactic tree kernels for the automatic judgment of the syntactic correctness of the questions. The questions are ranked by considering both their importance (in the context of the given body of texts) and syntactic correctness. To the best of our knowledge, no previous study has accomplished this task in our setting. A series of experiments demonstrate that the proposed topic-to-question generation approach can significantly outperform the state-of-the-art results.

Recurrent Convolutional Neural Networks for Text Classification

Text classification is a foundational task in many NLP applications. Traditional text classifiers often rely on many human-designed features, such as dictionaries, knowledge bases and special tree kernels. In contrast to traditional methods, we introduce a recurrent convolutional neural network for text classification without human-designed features. In our model, we apply a recurrent structure to capture contextual information as far as possible when learning word representations, which may introduce considerably less noise compared to traditional window-based neural networks. We also employ a max-pooling layer that automatically judges which words play key roles in text classification to capture the key components in texts. We conduct experiments on four commonly used datasets. The experimental results show that the proposed method outperforms the state-of-the-art methods on several datasets, particularly on document-level datasets.

Computation of Program Source Code Similarity by Composition of Parse Tree and Call Graph

This paper proposes a novel method to compute how similar two program source codes are. Since a program source code is represented as a structural form, the proposed method adopts convolution kernel functions as a similarity measure. Actually, a program source code has two kinds of structural information. One is syntactic information and the other is the dependencies of function calls lying on the program. Since the syntactic information of a program is expressed as its parse tree, the syntactic similarity between two programs is computed by a parse tree kernel. The function calls within a program provide a global structure of a program and can be represented as a graph. Therefore, the similarity of function calls is computed with a graph kernel. Then, both structural similarities are reflected simultaneously into comparing program source codes by composing the parse tree and the graph kernels based on a cyclomatic complexity. According to the experimental results on a real data set for program plagiarism detection, the proposed method is proved to be effective in capturing the similarity between programs. The experiments show that the plagiarized pairs of programs are found correctly and thoroughly by the proposed method.

Enhancing Predictive Analytics for Anti-Phishing by Exploiting Website Genre Information

Phishing websites continue to successfully exploit user vulnerabilities in household and enterprise settings. Existing anti-phishing tools lack the accuracy and generalizability needed to protect Internet users and organizations from the myriad of attacks encountered daily. Consequently, users often disregard these tools’ warnings. In this study, using a design science approach, we propose a novel method for detecting phishing websites. By adopting a genre theoretic perspective, the proposed genre tree kernel method utilizes fraud cues that are associated with differences in purpose between legitimate and phishing websites, manifested through genre composition and design structure, resulting in enhanced anti-phishing capabilities. To evaluate the genre tree kernel method, a series of experiments were conducted on a testbed encompassing thousands of legitimate and phishing websites. The results revealed that the proposed method provided significantly better detection capabilities than state-of-the-art anti-phishing methods. An additional experiment demonstrated the effectiveness of the genre tree kernel technique in user settings; users utilizing the method were able to better identify and avoid phishing websites, and were consequently less likely to transact with them. Given the extensive monetary and social ramifications associated with phishing, the results have important implications for future anti-phishing strategies. More broadly, the results underscore the importance of considering intention/purpose as a critical dimension for automated credibility assessment: focusing not only on the “what” but rather on operationalizing the “why” into salient detection cues.

Learning Structural Kernels for Natural Language Processing

Structural kernels are a flexible learning paradigm that has been widely used in Natural Language Processing. However, the problem of model selection in kernel-based methods is usually overlooked. Previous approaches mostly rely on setting default values for kernel hyperparameters or using grid search, which is slow and coarse-grained. In contrast, Bayesian methods allow efficient model selection by maximizing the evidence on the training data through gradient-based methods. In this paper we show how to perform this in the context of structural kernels by using Gaussian Processes. Experimental results on tree kernels show that this procedure results in better prediction performance compared to hyperparameter optimization via grid search. The framework proposed in this paper can be adapted to other structures besides trees, e.g., strings and graphs, thereby extending the utility of kernel-based methods.

An efficient topological distance-based tree kernel.

Tree kernels proposed in the literature rarely use information about the relative location of the substructures within a tree. As this type of information is orthogonal to the one commonly exploited by tree kernels, the two can be combined to enhance state-of-the-art accuracy of tree kernels. In this brief, our attention is focused on subtree kernels. We describe an efficient algorithm for injecting positional information into a tree kernel and present ways to enlarge its feature space without affecting its worst case complexity. The experimental results on several benchmark datasets are presented showing that our method is able to reach state-of-the-art performances, obtaining in some cases better performance than computationally more demanding tree kernels.

Kernel k-nearest neighbor classifier based on decision tree ensemble for SAR modeling analysis

Kernel approaches that can effectively solve nonlinear problems using implicit nonlinear mapping have been gaining popularity in the field of chemistry. In the present study, a novel tree kernel k-nearest neighbor algorithm (TKk-NN) has been proposed. First, an informative novel tree kernel is constructed based on the decision tree ensemble. The constructed tree kernel can effectively use important variables for classification and neglect useless variables through variable importance ranking during the process of building the kernel. Under the framework of kernel methods, this tree kernel is then extended to the k-nearest neighbor algorithm. Three SAR datasets together with the simulated data were used to test the performance of k-NN with tree and radial basis function kernels. The results show that TKk-NN really is an attractive alternative technique.