Multi-view Learning Framework Research Articles

Variable star classification using multiview metric learning

ABSTRACTComprehensive observations of variable stars can include time domain photometry in a multitude of filters, spectroscopy, estimates of colour (e.g. U-B), etc. When the objective is to classify variable stars, traditional machine learning techniques distill these various representations (or views) into a single feature vector and attempt to discriminate among desired categories. In this work, we propose an alternative approach that inherently leverages multiple views of the same variable star. Our multiview metric learning framework enables robust characterization of star categories by directly learning to discriminate in a multifaceted feature space, thus, eliminating the need to combine feature representations prior to fitting the machine learning model. We also demonstrate how to extend standard multiview learning, which employs multiple vectorized views, to the matrix-variate case which allows very novel variable star signature representations. The performance of our proposed methods is evaluated on the UCR Starlight and LINEAR data sets. Both the vector and matrix-variate versions of our multiview learning framework perform favourably – demonstrating the ability to discriminate variable star categories.

Multi-task and multi-view training for end-to-end relation extraction

Transfer learning has shown promising results for transferring knowledge ofsource tasks to target tasks in natural language processing (NLP). In this paper, we investigate a multi-task and multi-view learning (MTMVL) framework for end-to-end neural relation extraction, using large noisy data as one auxiliary task for improving manually labeled data, and a dependency parsing objective as a second auxiliary task for leveraging syntax. Two views are taken for relation extraction, consisting of a joint tagging view and a novel context relation view. To capture sentence multi-level information, we explore a weighted average approach to represent shared deep bi-directional recurrent neural networks and encourage auxiliary tasks to accommodate relation extraction task via a time-related scheduled sampling strategy. We evaluate MTMVL framework on manual-labeled ACE2005 dataset, and experimental results show that our model outperforms the state-of-the-art methods, which indicates the effectiveness of multi-auxiliary information for knowledge transferring.

Explainable Recommendation through Attentive Multi-View Learning

Recommender systems have been playing an increasingly important role in our daily life due to the explosive growth of information. Accuracy and explainability are two core aspects when we evaluate a recommendation model and have become one of the fundamental trade-offs in machine learning. In this paper, we propose to alleviate the trade-off between accuracy and explainability by developing an explainable deep model that combines the advantages of deep learning-based models and existing explainable methods. The basic idea is to build an initial network based on an explainable deep hierarchy (e.g., Microsoft Concept Graph) and improve the model accuracy by optimizing key variables in the hierarchy (e.g., node importance and relevance). To ensure accurate rating prediction, we propose an attentive multi-view learning framework. The framework enables us to handle sparse and noisy data by co-regularizing among different feature levels and combining predictions attentively. To mine readable explanations from the hierarchy, we formulate personalized explanation generation as a constrained tree node selection problem and propose a dynamic programming algorithm to solve it. Experimental results show that our model outperforms state-of-the-art methods in terms of both accuracy and explainability.

Letting Logos Speak: Leveraging Multiview Representation Learning for Data-Driven Logo Design

Logos serve a fundamental role as the visual figureheads of brands. Yet, due to the difficulty of using unstructured image data, prior research on logo design has largely been limited to non-quantitative studies. In this work, we explore the interplay between logo design and brand identity creation from a data-driven perspective. We develop both a novel logo feature extraction algorithm that uses modern image processing tools to decompose pixel-level image data into meaningful features, and a multiview representation learning framework that links these visual features to textual descriptions, consumer ratings of brand personality, and other high-level tags describing firms. We apply this framework to a unique dataset of brands, to understand which brands use which logo features, and how consumers evaluate these brands’ personalities. Moreover, we show that manipulating the model’s learned representations through what we term “brand arithmetic” yields new brand identities, and can help with ideation. Finally, through an application to fast food branding, we show how our model can be used as a decision support tool for suggesting typical logo features for a brand, and for predicting consumers’ reactions to new brands or rebranding efforts.

Hierarchy-Dependent Cross-Platform Multi-View Feature Learning for Venue Category Prediction

In this work, we focus on visual venue category prediction, which can facilitate various applications for location-based service and personalization. Considering that the complementarity of different media platforms, it is reasonable to leverage venue-relevant media data from different platforms to boost the prediction performance. Intuitively, recognizing one venue category involves multiple semantic cues, especially objects and scenes, and thus they should contribute together to venue category prediction. In addition, these venues can be organized in a natural hierarchical structure, which provides prior knowledge to guide venue category estimation. Taking these aspects into account, we propose a Hierarchy-dependent Cross-platform Multi-view Feature Learning (HCM-FL) framework for venue category prediction from videos by leveraging images from other platforms. HCM-FL includes two major components, namely Cross-Platform Transfer Deep Learning (CPTDL) and Multi-View Feature Learning with the Hierarchical Venue Structure (MVFL-HVS). CPTDL is capable of reinforcing the learned deep network from videos using images from other platforms. Specifically, CPTDL first trained a deep network using videos. These images from other platforms are filtered by the learnt network and these selected images are then fed into this learnt network to enhance it. Two kinds of pre-trained networks on the ImageNet and Places dataset are employed. MVFL-HVS is then developed to enable multi-view feature fusion. It is capable of embedding the hierarchical structure ontology to support more discriminative joint feature learning. We conduct the experiment on videos from Vine and images from Foursqure. These experimental results demonstrate the advantage of our proposed framework.

Perceptually learning multi-view sparse representation for scene categorization

Utilizing multi-channel visual features to characterize scenery images is standard for state-of-the-art scene recognition systems. However, how to encode human visual perception for scenery image modeling and how to optimally combine visual features from multiple views remains a tough challenge. In this paper, we propose a perceptual multi-view sparse learning (PMSL) framework to distinguish sceneries from different categories. Specifically, we first project regions from each scenery into the so-called perceptual space, which is established by combining human gaze behavior, color and texture. Afterward, a novel PMSL is developed which fuzes the above three visual cues into a sparse representation. PMSL can support absent channel visual features, which is frequently occurred in practical circumstances. Finally, the sparse representation from each scenery image is incorporated into an image kernel, which is further fed into a kernel SVM for scene categorization. Comprehensive experimental results on popular data sets have demonstrated the superiority of our method over well-known shallow/deep recognition models.

A Multi-View Deep Learning Framework for EEG Seizure Detection.

The recent advances in pervasive sensing technologies have enabled us to monitor and analyze the multi-channel electroencephalogram (EEG) signals of epilepsy patients to prevent serious outcomes caused by epileptic seizures. To avoid manual visual inspection from long-term EEG readings, automatic EEG seizure detection has garnered increasing attention among researchers. In this paper, we present a unified multi-view deep learning framework to capture brain abnormalities associated with seizures based on multi-channel scalp EEG signals. The proposed approach is an end-to-end model that is able to jointly learn multi-view features from both unsupervised multi-channel EEG reconstruction and supervised seizure detection via spectrogram representation. We construct a new autoencoder-based multi-view learning model by incorporating both inter and intra correlations of EEG channels to unleash the power of multi-channel information. By adding a channel-wise competition mechanism in the training phase, we propose a channel-aware seizure detection module to guide our multi-view structure to focus on important and relevant EEG channels. To validate the effectiveness of the proposed framework, extensive experiments against nine baselines, including both traditional handcrafted feature extraction and conventional deep learning methods, are carried out on a benchmark scalp EEG dataset. Experimental results show that the proposed model is able to achieve higher average accuracy and f1-score at 94.37% and 85.34%, respectively, using 5-fold subject-independent cross validation, demonstrating a powerful and effective method in the task of EEG seizure detection.

Multiview Latent Space Learning With Feature Redundancy Minimization.

Multiview learning has received extensive research interest and has demonstrated promising results in recent years. Despite the progress made, there are two significant challenges within multiview learning. First, some of the existing methods directly use original features to reconstruct data points without considering the issue of feature redundancy. Second, existing methods cannot fully exploit the complementary information across multiple views and meanwhile preserve the view-specific properties; therefore, the degraded learning performance will be generated. To address the above issues, we propose a novel multiview latent space learning framework with feature redundancy minimization. We aim to learn a latent space to mitigate the feature redundancy and use the learned representation to reconstruct every original data point. More specifically, we first project the original features from multiple views onto a latent space, and then learn a shared dictionary and view-specific dictionaries to, respectively, exploit the correlations across multiple views as well as preserve the view-specific properties. Furthermore, the Hilbert-Schmidt independence criterion is adopted as a diversity constraint to explore the complementarity of multiview representations, which further ensures the diversity from multiple views and preserves the local structure of the data in each view. Experimental results on six public datasets have demonstrated the effectiveness of our multiview learning approach against other state-of-the-art methods.

Multi-instance multi-label new label learning

In the studies of traditional multi-label learning, they assume the observed label distribution in the training data is the same as the true label distribution in the test data. In real applications, however, it is not the case; some existing semantics are not labeled throughout the dataset. In the testing phase, the learning system not only requires a good performance on the known target labels but also needs to detect the potential new labels. This paper proposes an end-to-end multi-view multi-instance multi-label learning approach, EM3NL, combining deep learning with multi-view multi-instance multi-label learning framework to address the new label learning problem. The performance on MS-COCO datasets validates the effectiveness of EM3NL on both the performance on the known labels and the new labels.

Adaptive-weighting discriminative regression for multi-view classification

Multi-view data represented by different features have been involved in many machine learning applications. Efficiently exploiting and preserving the correlative yet complementary information in multiple views remains challenging in multi-view learning. Comparing with existing methods that separately cope with each view, we propose a supervised multi-view feature learning framework to handle diverse views with a unified perception. Specifically, we fuse the multi-view data by mapping the concatenation of original features to a discriminative low-dimensional subspace, where the features from different views are adaptively assigned with the learned optimal weights. This strategy can simultaneously preserve the correlative and the complementary information, which is further enhanced to be more discriminative for subsequent classification. An efficient iterative algorithm is devised to optimize the formulated framework with closed-form solutions. Comprehensive evaluations with several state-of-the-art competitors demonstrate the efficiency and the superiority of the proposed method.

Low-Rank Multi-View Embedding Learning for Micro-Video Popularity Prediction

Recently, a prevailing trend of user generated content (UGC) on social media sites is the emerging micro-videos. Micro-videos afford many potential opportunities ranging from network content caching to online advertising, yet there are still little efforts dedicated to research on micro-video understanding. In this paper, we focus on popularity prediction of micro-videos by presenting a novel low-rank multi-view embedding learning framework. We name it as transductive low-rank multi-view regression (TLRMVR), and it is capable of boosting the performance of micro-video popularity prediction by jointly considering the intrinsic representations of the source and target samples. In particular, TLRMVR integrates low-rank multi-view embedding and regression analysis into a unified framework such that the lowest-rank representation shared by all views not only captures the global structure of all views, but also indicates the regression requirements. The framework is formulated as a regression model and it seeks a set of view-specific projection matrices with low-rank constraints to map multi-view features into a common subspace. In addition, a multi-graph regularization term is constructed to improve the generalization capability and further prevents the overfitting problem. Extensive experiments conducted on a publicly available dataset demonstrate that our proposed method achieve promising results as compared with state-of-the-art baselines.

A two-stage multi-view learning framework based computer-aided diagnosis of liver tumors with contrast enhanced ultrasound images.

With the fast development of artificial intelligence techniques, we proposed a novel two-stage multi-view learning framework for the contrast-enhanced ultrasound (CEUS) based computer-aided diagnosis for liver tumors, which adopted only three typical CEUS images selected from the arterial phase, portal venous phase and late phase. In the first stage, the deep canonical correlation analysis (DCCA) was performed on three image pairs between the arterial and portal venous phases, arterial and delayed phases, and portal venous and delayed phases respectively, which then generated total six-view features. While in the second stage, these multi-view features were then fed to a multiple kernel learning (MKL) based classifier to further promote the diagnosis result. Two MKL classification algorithms were evaluated in this MKL-based classification framework. We evaluated proposed DCCA-MKL framework on 93 lesions (47 malignant cancers vs. 46 benign tumors). The proposed DCCA-MKL framework achieved the mean classification accuracy, sensitivity, specificity, Youden index, false positive rate, and false negative rate of 90.41 ± 5.80%, 93.56 ± 5.90%, 86.89 ± 9.38%, 79.44 ± 11.83%, 13.11 ± 9.38% and 6.44 ± 5.90%, respectively, by soft margin MKL classifier. The experimental results indicate that the proposed DCCA-MKL framework achieves best performance for discriminating benign liver tumors from malignant liver cancers. Moreover, it is also proved that the three-phase CEUS image based CAD is feasible for liver tumors with the proposed DCCA-MKL framework.

SAR Automatic Target Recognition Based on Multiview Deep Learning Framework

It is a feasible and promising way to utilize deep neural networks to learn and extract valuable features from synthetic aperture radar (SAR) images for SAR automatic target recognition (ATR). However, it is too difficult to effectively train the deep neural networks with limited raw SAR images. In this paper, we propose a new approach to do SAR ATR, in which a multiview deep learning framework was employed. Based on the multiview SAR ATR pattern, we first present a flexible mean to generate adequate multiview SAR data, which can guarantee a large amount of inputs for network training without needing many raw SAR images. Then, a unique deep convolutional neural network containing a parallel network topology with multiple inputs is adopted. The features of input SAR images from different views will be learned by the proposed network layer by layer; meanwhile, the learned features from the distinct views are fused in different layers progressively. Therefore, the proposed framework is able to achieve a superior recognition performance, and requires only a small number of raw SAR images for network training samples generation. Experimental results have shown the superiority of the proposed framework based on the Moving and Stationary Target Acquisition and Recognition data set.

Real-Time Human Mobility Modeling with Multi-View Learning

Real-time human mobility modeling is essential to various urban applications. To model such human mobility, numerous data-driven techniques have been proposed. However, existing techniques are mostly driven by data from a single view, for example, a transportation view or a cellphone view, which leads to over-fitting of these single-view models. To address this issue, we propose a human mobility modeling technique based on a generic multi-view learning framework called coMobile. In coMobile, we first improve the performance of single-view models based on tensor decomposition with correlated contexts, and then we integrate these improved single-view models together for multi-view learning to iteratively obtain mutually reinforced knowledge for real-time human mobility at urban scale. We implement coMobile based on an extremely large dataset in the Chinese city Shenzhen, including data about taxi, bus, and subway passengers along with cellphone users, capturing more than 27 thousand vehicles and 10 million urban residents. The evaluation results show that our approach outperforms a single-view model by 51% on average. More importantly, we design a novel application where urban taxis are dispatched based on unaccounted mobility demand inferred by coMobile.

A Multiview Learning Framework With a Linear Computational Cost.

Learning features from multiple views has attracted much research attention in different machine learning tasks, such as multiclass and multilabel classification problems. In this paper, we propose a multiclass multilabel multiview learning framework with a linear computational cost where an example is associated with at least one label and represented by multiple information sources. We simultaneously analyze all features by learning an integrated projection matrix. We can also automatically select more important views for subsequent classifier to predict each class. As the proposed objective function is nonsmooth and difficult to solve, we apply a novel optimization method that converts the multiview learning problem to a set of linear single-view learning problems by bridging our problem to an easily solvable approach. Compared to the conventional methods which learn the entire projection matrix, our algorithm independently optimizes each column of the projection matrix for each class, which can be easily parallelized. In each column optimization, the most computationally intensive step is pure and simple matrix-by-vector multiplication. As a result, our algorithm is much more applicable to large-scale problems than the multiview learning methods with a nonlinear computational cost. Moreover, rigorous convergence proof of the proposed algorithm is also provided. To evaluate the effectiveness of the proposed approach, experimental comparisons are made with state-of-the-art algorithms in multiclass and multilabel classification tasks on many multiview benchmarks. We also report the efficiency comparison results on different numbers of data samples. The experimental results demonstrate that our algorithm can achieve superior performance to all the compared algorithms.

Top-k multi-class SVM using multiple features

Recent studies have demonstrated the advantages of fusing multi-modal features in improving the accuracy of visual object classification. However, regarding a complex classification task with a large number of categories, previous studies on multiple feature fusion are prone to failure resulting from the occurrence of class ambiguity. In this paper, we address this issue by allowing k (k ≥ 2) guesses at the top instead of only considering the one with the largest prediction score in the framework of multi-view learning. This strategy relaxes the penalty for making an error in the top-k predictions, which can mitigate the challenge of class ambiguity to some extent. To fuse multiple features effectively, we introduce an adaptive weight for each view and exploit an efficient alternating optimization algorithm to learn the optimal classifiers and their corresponding weights jointly. Extensive experiments on several benchmark datasets illustrate the effectiveness and superiority of the proposed model over the state-of-the-art approaches.

Multimodal Ultrasound imaging based diagnosis of liver cancers with a two-stage multi-view learning framework.

Computer-aided diagnosis (CAD) of liver cancers on contrast-enhanced ultrasound (CEUS) has attracted considerable attention in recent years. The enhancement patterns on CEUS for liver lesions consist of the arterial, portal venous and late phases. Several typical images selected from these three phases can provide reliable information basis for diagnosis of liver lesions. Therefore, we propose to develop a CAD framework for liver cancers with only one B-mode image and three typical CEUS images selected from three enhancement patterns, which simulates the clinical diagnosis mode of radiologists. Moreover, a framework of two-stage multi-view learning (TS-MVL) is proposed to perform both feature-level and classifier-level MVL for the diagnosis of liver cancers with multimodal ultrasound images. We propose to apply the nonlinear kernel matrix (NKM) algorithm to effectively fuse the features of multimodal ultrasound images, and then perform the multiple kernel boosting (MKB) algorithm to promote the predictive performance of multiple classifiers according to multi-view features. The experimental results indicate that the proposed algorithm outperforms the commonly used multi-view learning algorithms.

Recognition of Epileptic EEG Signals Using a Novel Multiview TSK Fuzzy System

Recognition of epileptic electroencephalogram (EEG) signals using machine learning techniques is becoming popular. In general, the construction of intelligent epileptic EEG recognition system involves two steps. First, an appropriate feature extraction method is applied to obtain representative features from the original raw EEG signals. Second, an effective intelligent model is trained based on the extracted features. However, there exist two major challenges in the process: 1) it is nontrivial to determine the appropriate feature extraction method to be used; 2) although many classical machine learning methods have been used for epileptic EEG recognition, most of them are “black box” approaches and more interpretable methods are desirable. To address these two challenges, a new epileptic EEG recognition method based on a multiview learning framework and fuzzy system modeling is proposed. First, multiview EEG data are generated by employing different feature extraction methods to obtain the features from different views of the signals. Second, the classical Takagi-Sugeno-Kang fuzzy system (TSK-FS) is introduced as an easy-to-interpret recognition model to develop a multiview TSK-FS method, called MV-TSK-FS, to identify epileptic EEG signals. For the proposed MV-TSK-FS, the importance of each view, i.e., the importance of each feature extraction method, can be evaluated according to the weighting of each view, and consequently the final decision can be made based on the weighted outputs of different views. Experimental results indicate that the MV-TSK-FS is a promising method when compared with the state-of-the-art algorithms.

Retargeted Multi-View Feature Learning With Separate and Shared Subspace Uncovering

Multi-view feature learning aims at improving the performances of learning tasks, by fusing various kinds of features (views), such as heterogeneous features and/or homogeneous features. Current leading multi-view feature learning approaches usually learn features in each view separately while not uncovering shared information from multiple views. In this paper, we propose a multi-view feature learning framework, which can simultaneously learn separate subspace for each view and shared subspace for all the views, respectively; specifically, the separate subspace for each view can preserve the particular information within this view, meanwhile, the shared subspace can capture feature correlation among multiple views. Both the particularity and communality are essential for classification. Furthermore, we relax the labels of training samples within the concatenated subspaces, thus resulting in the retargeted least square regression (LSR) classifier. The transformation matrices tailored for each subspace within the corresponding view and the label relaxed LSR classifier are jointly learned in a unified framework, based on an efficient alternative optimization manner. Extensive experiments on four benchmark data sets well demonstrate the superiority of the proposed method, which has led to better performances than compared counterpart methods.

A Robust and Versatile Multi-View Learning Framework for the Detection of Deviant Business Process Instances

Increasing attention has been paid to the detection and analysis of “deviant” instances of a business process that are connected with some kind of “hidden” undesired behavior (e.g. frauds and faults). In particular, several recent works faced the problem of inducing a binary classification model (here named deviance detection model ) that can discriminate between deviant traces and normal ones, based on a set of historical log traces (labeled as either deviant or normal). Current solutions rely on applying standard classifier-induction methods to a feature-based representation of the given traces, where the features include sequence-based patterns extracted from the corresponding sequences of activities. However, there is no consensus on which kinds of patterns are the most suitable for such a task. On the other hand, mixing multiple pattern families together may produce a heterogenous, redundant and sparse representation of the traces that likely leads to poor deviance detection models. In this paper, we propose an ensemble-learning method for solving this problem, where multiple base classifiers are trained on different feature-based views of the log (each obtained by mapping the traces onto a distinguished collection of patterns). A stacking procedure is used to combine the discovered base models into an overall probabilistic model that associates any new trace with an estimate of the probability that it reflects a deviant process instance. This helps the analyst prioritize the inspection of the cases that are more likely to be deviant. The method also takes advantage of all nonstructural data available in the log, and employs a resampling mechanism to deal with the rarity of deviances in the training log. It has been conceived as the core of a comprehensive framework for detecting and analyzing business process deviances. The framework supports the analyst to investigate suspect deviances, and provides some feedback to the learning method for improving the accuracy of the discovered deviance detection models. Tests on several real-life datasets proved the validity of the approach, as concerns its capability to discover an accurate deviance detection model, and to effectively exploit new (originally unlabeled) traces via active learning and self-training mechanisms.