Tree-structured Learning Research Articles

Improving Uncertainty Quantification of Variance Networks by Tree-Structured Learning.

To improve the uncertainty quantification of variance networks, we propose a novel tree-structured local neural network model that partitions the feature space into multiple regions based on uncertainty heterogeneity. A tree is built upon giving the training data, whose leaf nodes represent different regions where region-specific neural networks are trained to predict both the mean and the variance for quantifying uncertainty. The proposed uncertainty-splitting neural regression tree (USNRT) employs novel splitting criteria. At each node, a neural network is trained on the full data first, and a statistical test for the residuals is conducted to find the best split, corresponding to the two subregions with the most significant uncertainty heterogeneity between them. USNRT is computationally friendly, because very few leaf nodes are sufficient and pruning is unnecessary. Furthermore, an ensemble version can be easily constructed to estimate the total uncertainty, including the aleatory and epistemic. On extensive UCI datasets, USNRT or its ensemble shows superior performance compared to some recent popular methods for quantifying uncertainty with variances. Through comprehensive visualization and analysis, we uncover how USNRT works and show its merits, revealing that uncertainty heterogeneity does exist in many datasets and can be learned by USNRT.

Multi-View Tree Structure Learning for 3D Model Retrieval and Classification in Smart City

The application of digital products in smart city results in ever-increasing 3D model data and how to obtain the relevant 3D model becomes a crucial issue. In this paper, we propose the Multi-View Tree Structure (MVTS) learning for 3D model retrieval and recognition. MVTS contains three key consecutive modules. Firstly, the visual feature learning module extracts the visual features of multiple views. Then, we design a score matrix to estimate the value of contextual information between view pairs. Based on the score matrix, a maximum spanning tree is constructed to further explore the contextual information within multiple views. Then, we utilize the bidirectional Tree-LSTM to encode the contextual information among views and the spatial information of tree structure and optimize the tree parameters. After that, the tree attention strategy is adopted to explore the importance of each view. Comparing to existing methods, our proposed method explores the spatial information of 3D model without the requirement of specific camera settings, which is more suitable for real applications. Moreover, our method jointly realizes the feature learning, view-wise contextual information and tree spatial information encoding and view importance estimating, which enhances the discrimination of the 3D model representation. Extensive experimental results on Modelnet40 and ShapeNetCore55 demonstrate the superiority of our method.

Cross recurrence quantifiers as new connectivity measures for structure learning of Bayesian networks in brain decoding

Abstract Bayesian networks were efficiently applied for brain decoding along with connectivity information used in structure learning of Bayesian networks. The modified structure learning proposed expands the application of Bayesian networks in brain-decoding. There are various structure learning methods which usually require large training samples. As the simplest method, tree structure learning commonly uses measures like correlation or mutual information to quantify dependencies in a network. Correlation captures the linear relations and mutual information detects linear and nonlinear dependencies while it needs more data to estimate the distribution of the variables. In current study, these measures were substituted with cross recurrence quantifiers which efficiently describes nonlinear behaviors of biological systems and effectively detect linear and nonlinear interactions without requiring long length of data. As novel applications, we used cross recurrence quantifiers for structure learning of Bayesian networks and functional connectivity analysis of fMRI data. To compare the performance of cross recurrence quantifiers with conventional indices in learning the structure of Bayesian networks, we used them in retrieving known structures. Moreover, the new method was applied for extracting fMRI brain connections in Bayesian networks devised for decoding-reconstruction of hand written digits. Objective and subjective assessment of results showed that Bayesian networks which used correlation or mutual information yielded significantly inferior accuracy in comparison to Bayesian networks which employed cross recurrence quantifiers. Therefore, cross recurrence quantifiers could be efficient measures in structure learning of Bayesian networks or functional connectivity analysis of fMRI data.

Fault detection and diagnosis for building cooling system with a tree-structured learning method

In order to save energy and improve the performance of building environment regulation, there is an increasing need for fault detection and diagnosis (FDD). This paper investigates the effectiveness of tree-structured learning method for FDD of building cooling system. Researchers have been tackling building FDD task with a wide variety of techniques, such as analytical model-based, signal-based and knowledge-based methods. Recently data-driven method has shown its advantage in dealing with complex systems with random penetrations. Existing work on data-driven FDD merely formulates the task as a pure fault type classification problem, whereas fault severity levels and their inter-dependence have long been ignored. We propose a novel data-driven strategy that adopts structured labeling to include the dependence information and describe the severity levels in a large margin learning framework. A Tree-structured Fault Dependence Kernel (TFDK) method is derived and a corresponding on-line learning algorithm is developed for streaming data. As an improvement of traditional classification methods (e.g. SVM), TFDK encodes tree-structured fault dependence in its feature mapping, and takes regularized misclassification cost as learning objective. Following the ASHRAE Research Project 1043 (RP-1043), the strategy is applied to the FDD of a 90-ton centrifugal water-cooled chiller. Experimental results show that compared to previous data-driven methods, TFDK can greatly improve the FDD performance as well as recognize the fault severity levels with high accuracy.

A Fuzzy Tree Matching-Based Personalized E-Learning Recommender System

The rapid development of e-learning systems provides learners with great opportunities to access learning activities online, and this greatly supports and enhances the learning practices. However, an issue reduces the success of application of e-learning systems; too many learning activities (such as various leaning materials, subjects, and learning resources) are emerging in an e-learning system, making it difficult for individual learners to select proper activities for their particular situations/requirements because there is no personalized service function. Recommender systems, which aim to provide personalized recommendations for products or services, can be used to solve this issue. However, e-learning systems need to be able to handle certain special requirements: 1) leaning activities and learners’ profiles often present tree structures; 2) learning activities contain vague and uncertain data, such as the uncertain categories that the learning activities belong to; 3) there are pedagogical issues, such as the precedence relations between learning activities. To deal with the three requirements, this study first proposes a fuzzy tree-structured learning activity model, and a learner profile model to comprehensively describe the complex learning activities and learner profiles. In the two models, fuzzy category trees and related similarity measures are presented to infer the semantic relations between learning activities or learner requirements. Since it is impossible to have two completely same trees, in practice, a fuzzy tree matching method is carefully discussed. A fuzzy tree matching-based hybrid learning activity recommendation approach is then developed. This approach takes advantage of both the knowledge-based and collaborative filtering-based recommendation approaches, and considers both the semantic and collaborative filtering similarities between learners. Finally, an e-learning recommender system prototype is well designed and developed based on the proposed models and recommendation approach. Experiments are done to evaluate the proposed recommendation approach, and the experimental results demonstrate the good accuracy performance of the proposed approach. A comprehensive case study about learning activity recommendation further demonstrates the effectiveness of the fuzzy tree matching-based personalized e-learning recommender system in practice.

Dependence tree structure estimation via copula

We propose an approach for dependence tree structure learning via copula. A nonparametric algorithm for copula estimation is presented. Then a Chow-Liu like method based on dependence measure via copula is proposed to estimate maximum spanning bivariate copula associated with bivariate dependence relations. The main advantage of the approach is that learning with empirical copula focuses on dependence relations among random variables, without the need to know the properties of individual variables as well as without the requirement to specify parametric family of entire underlying distribution for individual variables. Experiments on two real-application data sets show the effectiveness of the proposed method.

Complexity-Regularized Tree-Structured Partition for Mutual Information Estimation

A new histogram-based mutual information estimator using data-driven tree-structured partitions (TSP) is presented in this paper. The derived TSP is a solution to a complexity regularized empirical information maximization, with the objective of finding a good tradeoff between the known estimation and approximation errors. A distribution-free concentration in equality for this tree-structured learning problem as well as finite sample performance bounds for the proposed histogram-based solution is derived. It is shown that this solution is density-free strongly consistent and that it provides, with an arbitrary high probability, an optimal balance between the mentioned estimation and approximation errors. Finally, for the emblematic scenario of independence, I(X; Y) = 0, it is shown that the TSP estimate converges to zero with o(e-n1/3+ log log n).