Multiple Feature Views Research Articles

Multiview Multilabel Classification With Group-Based Feature and Label Selection

In real-world applications, data are often represented by multiple feature views and associated with multiple labels. In multiview learning, as different views may have noisy and irrelevant features, many works target multiview feature selection, but most of them only perform global feature selection, which means that all samples share the same feature selection weights. In addition, there are many studies in multilabel learning that assume all samples share the same label correlation. However, data may exhibit local patterns, such as feature selection weights and label correlations are locally shared by samples. To address this issue, in this paper, we propose a novel group-based model with local feature and label selection. The proposed model can project samples into different groups by performing group-based feature selection with each view having its own importance for grouping, where each group has its own related labels. The proposed model can then predict the semantics of samples by performing group-based label selection with each group having its own weight for prediction. Besides, the inter-group correlation is also mined and introduced in the above group-based learning process to ensure effective multilabel classification. Empirical studies on multiple image benchmarks validate the effectiveness of the proposed group-based model.

A new multi-view multi-label model with privileged information learning

In multi-view multi-label learning (MVML), the data is described by multiple feature views and annotated by a number of categorical labels. At present, most of the existing MVML methods are proposed based on subspace learning, neural networks and so on. There is little work done on support vector machine (SVM)-based MVML. In this paper, we propose a novel SVM-based multi-view multi-label learning method with privileged information learning (MVMLP). By introducing the idea of privileged information learning, MVMLP implements both the consensus principle and complementarity principle in MVML. Specifically, we enforce similarity between the outputs of different views to explore the consensus information. Moreover, we allow different views to serve as privileged information for each other, such that the complementary information among distinct views can be introduced into the training process. MVMLP constructs an SVM-based model for each view and trains the models of multiple views jointly. The derived learning problem can be efficiently solved by adapting the Franke-Wolfe algorithm. Experimental results on real-life datasets show that MVMLP delivers explicitly better classification performance than state-of-the-art MVML methods.

Fast metric multi-view hashing for multimedia retrieval

The acquisition of multi-view hash representation for heterogeneous data holds paramount importance in the domain of multimedia retrieval. The limited retrieval precision observed in current approaches stems from their inadequate integration of multi-view features and their failure to effectively leverage the metric information available from diverse samples. Commonly employed fusion methods, such as concatenation or weighted sum, are insufficient in capturing the complementarity among multiple view features. Furthermore, these methods neglect the valuable information contributed by dissimilar samples. To address these challenges, we propose an innovative method termed Fast Metric Multi-View Hashing (FMMVH). Our approach showcases the superiority of gate-based fusion over traditional methods, as substantiated by extensive empirical evidence. Additionally, this paper proposes a novel deep metric loss function to enable the utilization of metric information from dissimilar samples. We exclusively train our method using this single loss function. To enhance practical applicability in industrial production environments, we employ model compression techniques to optimize the proposed method. On benchmark datasets such as MIR-Flickr25K, NUS-WIDE, and MS COCO, the performance of our FMMVH method significantly surpasses that of existing state-of-the-art methods, demonstrating improvements of up to 7.47% in mean Average Precision (mAP).

SenticGAT: Sentiment Knowledge Enhanced Graph Attention Network for Multi-view Feature Representation in Aspect-based Sentiment Analysis

Currently, computational intelligence methods, especially artificial neural networks, are increasingly applied to many scenarios. We mainly attempt to explore the task of fine-grained sentiment classification of review data through computational intelligence methods, especially artificial neural networks, and this task is also known as aspect-based sentiment analysis (ABSA). We propose a new technique called SenticGAT which is a multi-view features fusion model enhanced by an external sentiment database. We encode the external sentiment information into the syntactic dependency tree to obtain an enhanced graph with rich sentiment representation. Then we obtain multi-view features including semantics, syntactic, and sentiment features through GAT based on the enhanced graph by external knowledge. We also design a new strategy for fusing multi-view features using the feature parallel frame and convolution method. Eventually, the sentiment polarity of a specific aspect is determined based on the completely fused multi-view features. Experimental results on four public benchmark datasets demonstrate that our method is effective and sound. And it performs superiorly to existing approaches in fusion multiple-view features.

Inclusivity induced adaptive graph learning for multi-view clustering

Graph-based multi-view clustering, with its ability to mine potential associations between data samples, has attracted extensive attention. However, existing methods directly learn affinity graphs from multiple feature views without removing noisy information mixed with the original data, resulting in limited performance. In addition, only shared information is included in the graph fusion process, which excludes inconsistent information, resulting in insufficient exploration of complementary information among multi-view data. In this study, we propose an inclusivity-induced adaptive graph learning (IiAGL) method to address these issues. First, inclusivity is defined as the capability of a view to accommodate other views. Specifically, we extended subspace learning to graph construction, enabling view-specific graphs to capture the underlying data distribution. Furthermore, an inclusivity-induced regularizer is introduced to guide graph rebuilding and graph fusion, and the entire process is performed using a self-weighted strategy. In this manner, we can obtain a clean consensus containing shared representations and complementary information from different views. An alternating direction method with augmented Lagrangian multiplier (ADM-ALM) was designed to solve the resulting optimization problem. Extensive experiments on diverse datasets demonstrated that the clustering performance of the proposed method outperformed that of other state-of-the-art methods. The source code for this study is publicly available at https://github.com/ryan-xinzou/IiAGL-MC.

Multi-view underwater image enhancement method via embedded fusion mechanism

Due to wavelength-dependent light absorption and scattering, underwater images often appear with a colour cast and blurry details. Most existing deep learning methods utilize a single input end-to-end network structure, which leads to a single form and content of the extracted features. To address these problems, we present a novel multi-feature underwater image enhancement method via embedded fusion mechanism (MFEF). We find that the quality of reconstruction results is affected by the quality of the input image to some extent, and use pre-processing to obtain high-quality images, which can improve the final reconstruction effect. We introduce the white balance (WB) algorithm and the contrast-limited adaptive histogram equalization (CLAHE) algorithm employing multiple path inputs to extract different forms of rich features in multiple views. To fully interact with features from multiple views, we design a multi-feature fusion (MFF) module to fuse derived image features. We suggest a novel pixel-weighted channel attention module (PCAM) that calibrates the detailed features of the degraded images using a weight matrix to give diverse weights to the encoded features. Ultimately, our network utilizes a fusion mechanism-based encoder and decoder that can be applied to restore various underwater scenes. In the UIEB dataset, our PSNR increased by 10.2% compared to that of Ucolor. Extensive experimental results demonstrate that the MFEF method outperforms other state-of-the-art underwater image enhancement methods in various real-world datasets.

3D multi-view squeeze-and-excitation convolutional neural network for lung nodule classification.

Early screening is crucial to improve the survival rate and recovery rate of lung cancer patients. Computer-aided diagnosis system (CAD) is a powerful tool to assist clinicians in early diagnosis. Lung nodules are characterized by spatial heterogeneity. However, many attempts use the two-dimensional multi-view (MV) framework to learn and simply integrate multiple view features. These methods suffer from the problems of not capturing the spatial characteristics effectively and ignoring the variability of multiple views. In this paper, we propose a three-dimensional MV convolutional neural network (3D MVCNN) framework and embed the squeeze-and-excitation (SE) module in it to further address the variability of each view in the MV framework. First, the 3D multiple view samples of lung nodules are extracted by the spatial sampling method, and a 3D CNN is established to extract 3D abstract features. Second, build a 3D MVCNN framework according to the 3D multiple view samples and 3D CNN. This framework can learn more features of different views of lung nodules, taking into account the characteristics of spatial heterogeneity of lung nodules. Finally, to further address the variability of each view in the MV framework, a 3D MVSECNN model is constructed by introducing a SE module in the feature fusion stage. For training and testing purposes we used independent subsets of the public LIDC-IDRI dataset. For the LIDC-IDRI dataset, this study achieved 96.04% accuracy and 98.59% sensitivity in the binary classification, and 87.76% accuracy in the ternary classification, which was higher than other state-of-the-art studies. The consistency score of 0.948 between the model predictions and pathological diagnosis was significantly higher than that between the clinician's annotations and pathological diagnosis. The results show that our proposed method can effectively learn the spatial heterogeneity of nodules and solve the problem of multiple view variability. Moreover, the consistency analysis indicates that our method can provide clinicians with more accurate results of benign-malignant lung nodule classification for auxiliary diagnosis, which is important for assisting clinicians in clinical diagnosis.

Mlo and CC view of feature fusion and mammogram classification using deep convolution neural network

Breast cancer is the most frequent type of cancer in women all over the world. The improvement of computer aided system help the radiologist for the effective analysis and diagnosis of breast cancer. It presents a computational methodology for classifying breast cancer as normal, benign and malignant from CC and MLO views of mammogram image. The proposed strategy consists of feature extraction, multiple view feature fusion and classification. The input images are fed into feature extraction where convolution neural network is applied. The CNN is nicely suitable for both feature extraction, feature fusion and mammogram classification. In this framework, convolution layer, pooling and activation function are used as a feature extraction techniques. After the process of feature extraction, feature fusion is employed by average pooling of CNN. The feature fusion will increase or maximize the relevant information of the breast image. Finally obtained features from the fusion are fed into CNN classifier in which softmax and fully connected layer are employed as a classifier techniques. The proposed work achieves 98.4% of accuracy to classify the breast cancer from MLO and CC views using hybrid feature with CNN classifier.

ASM2TV: An Adaptive Semi-supervised Multi-Task Multi-View Learning Framework for Human Activity Recognition

Many real-world scenarios, such as human activity recognition (HAR) in IoT, can be formalized as a multi-task multi-view learning problem. Each specific task consists of multiple shared feature views collected from multiple sources, either homogeneous or heterogeneous. Common among recent approaches is to employ a typical hard/soft sharing strategy at the initial phase separately for each view across tasks to uncover common knowledge, underlying the assumption that all views are conditionally independent. On the one hand, multiple views across tasks possibly relate to each other under practical situations. On the other hand, supervised methods might be insufficient when labeled data is scarce. To tackle these challenges, we introduce a novel framework ASM2TV for semi-supervised multi-task multi-view learning. We present a new perspective named gating control policy, a learnable task-view-interacted sharing policy that adaptively selects the most desirable candidate shared block for any view across any task, which uncovers more fine-grained task-view-interacted relatedness and improves inference efficiency. Significantly, our proposed gathering consistency adaption procedure takes full advantage of large amounts of unlabeled fragmented time-series, making it a general framework that accommodates a wide range of applications. Experiments on two diverse real-world HAR benchmark datasets collected from various subjects and sources demonstrate our framework's superiority over other state-of-the-arts. Anonymous codes are available at https://github.com/zachstarkk/ASM2TV.

When multi-view classification meets ensemble learning

With the coming of big data era, multi-view data represented by multiple features have been involved in many terms, such as machine learning, data mining and computer vision and so on. Due to the complex structure hidden in data, how to utilize the complementary and correlative information among multiple view features to improve classification performance is a challenging task. Moreover, the another challenging task is how to assign an appropriate weight for each classifier on the basis of its performance. To solve above problems, we proposed a supervised multi-view classification method based on Least Square Regression (LSR) and Ensemble Learning. To be specific, all samples for each view firstly can be classified by using Multi-class Support Vector Machine (MSVM); Then, to evaluate the classification results of different views for each sample, the optimal weight of each sample classification result is learned; Furthermore, considering the view difference with different classification quality, the view weights are assigned adaptively; Finally, we adopt the decision function values to determine the final classification results. Extensive experimental results show that the proposed method outperforms most state-of-the-art multi-view classification methods.

A composite framework of deep multiple view human joints feature extraction and selection strategy with hybrid adaptive sunflower optimization‐whale optimization algorithm for human action recognition in video sequences

AbstractIn computer vision and pattern recognition field, video‐based human action recognition (HAR) is the most predominant research area. Object recognition is needed to recognize the subjects regarding video contents, which allows reactive enquiry in a large number of camera contents, mainly in security based platforms where there is a prevalent growth of closed circuit television cameras. Generally, object detectors that have high performance are trained on a large collection of public benchmarks. Identifying human activities from unconstrained videos is the primary challenging task. Further, the feature extraction and feature selection from these unconstrained videos is also considered as a challenging issue. For that, in this article a new composite framework of HAR model is constructed by introducing an efficient feature extraction and selection strategy. The proposed feature extraction model extracts multiple view features, human joints features based on the domain knowledge of the action and fuses them with deep high level features extracted by an improved fully resolution convolutional neural networks. Also, it optimizes the feature selection strategy using the hybrid whale optimization algorithm and adaptive sun flower optimization that maximizes the feature entropy, correlation. It minimizes the error rate for improving the recognition accuracy of the proposed composite framework. The proposed model is validated on four different datasets, namely, Olympics sports, Virat Release 2.0, HMDB51, and UCF 50 sports action dataset to prove its effectiveness. The simulation results show that the proposed composite framework outperforms all the existing human recognition model in terms of classification accuracy and detection rate.

DFT: A deep feature‐based semi‐supervised collaborative training for vehicle recognition in smart cities

AbstractIntelligent transport systems (ITS) is a popular field of research in smart city. Image‐based vehicle recognition is one of the most promising new techniques. The traditional appearance‐based method has some limitations in feature view. The machine learning algorithm can satisfy multiple feature views, but it mainly adopts an improved supervised learning method, which needs manual annotation and has low efficiency. The semi‐supervised collaborative training method is widely used in the image recognition process to improve machine learning algorithms' accuracy and generalization performance. This paper proposes DFT (deep feature‐based training) method for vehicle recognition in smart cities. DFT is also a semi‐supervised collaborative training method on basis of two base learners. DFT adjusts data pre‐processing and training process, optimizes the constructing a disagreement encoding network, and expands the recognition disagreement of pseudo‐labelled samples‐based training set. Compared with the typical collaborative training methods, DFT greatly accelerates the model's training process by reducing the convergence time, and improves the efficiency of vehicle recognition, while remaining the recognition accuracy unchanged.

Self-weighting multi-view spectral clustering based on nuclear norm

Multi-view clustering attracts more and more attention due to the fact that it can utilize the complementary and compatible information from multi-view data sets. In many graph-based multi-view clustering approaches, the graph quality is important since it influences the following clustering performance. Therefore, learning a high quality similarity graph is desired. In this paper, we propose a novel clustering method which is named as Self-weighting Multi-view Spectral Clustering based on Nuclear Norm (SMSC_NN). Specifically, to fully utilize the multiple view features, the common consensus representation is learned. Moreover, to capture the principal components from various view features, the nuclear norm is introduced which can make the view-specific information be well explored. Further, due to the fact that each view feature denotes a sort of specific property, the adaptive weights are assigned instead of equal view weights. In order to verify the effectiveness of the proposed method, four multi-view data sets are used to conduct the clustering experiments. Extensive experimental results demonstrate the superiority of the proposed method comparing with state-of-the-art multi-view clustering approaches. In addition, the proposed approach is experimented on the Cal101-20 data set with ”salt and pepper” noises, and experimental results verify that the proposed SMSC_NN method can remain robust to noises.

Efficient Feature Extraction from Multispectral Images for Face Recognition Applications: A Deep Learning Approach

In recent years many face recognition algorithms were used for the identification and authentication of a person to a system. However, still, feature extraction from multispectral images was considered to be a challenging task. Feature extraction, including highlight location and portrayal, assumes a significant job in real-time security-based applications. In this paper, a novel Geometric Algebra-based Multivariate Regression Feature Extraction (GA-MVRFE) algorithm was proposed to extract features from a huge dataset stored in the cloud efficiently. This proposed algorithm works with the supreme expedient deep learning approach - Convolutional Neural Network (CNN) for image classification. CNN will automatically detect significant features from the multispectral images without any human intrusion from a huge database. Real-time images were captured with three different cameras and applied filters over the images and were created as a dataset. To show the competence of the proposed algorithm, an exclusively created dataset with a set of 14,400 image data was applied in the proposed and other existing algorithms, and their efficiency and robustness were noted. Providentially, GA-MVRFE produced better accuracy in ‘Face Recognition’ with a less time fraction compared with former algorithms. Obtained accuracy % for Geometric Algebra Oriented fast and Rotated Brief (GA-ORB), Geometric Algebra Fast Retina key-point Extraction Algorithm (GA-FREAK), Trilateral Smooth Filtering (TRSF), Cross Regression Multiple View Features extraction (CRMVF) and GA-MVRFE was 87.81, 83.23, 90.72, 91.67 and 97.57 respectively.

Two-step multi-view and multi-label learning with missing label via subspace learning

In multi-view and multi-label learning, each example can be represented by multiple data view features and annotated with a set of discrete non-exclusive labels. Missing label learning is an important branch of multi-label learning, which can handle incomplete labels with annotations. Previous work on multi-label learning with missing labels mainly considered data in a single view representation. Based on intuitive understanding, we propose a Two-step Multi-view and Multi-label Missing Label learning optimization solution(TM3L). The first step is to solve the multi-view learning problem by finding the data representation of the common low-dimensional space of all views through subspace learning. While fully considering the complementary information between multiple views, the different degrees of contribution combined with different views are weighted differently. The second step is to solve the multi-label missing label learning problem by using the label matrix completion method in combination with the kernel extreme learning machine classifier. The kernel extreme learning machine can effectively enhance the robustness of the algorithm to missing labels. The experimental results and analysis on multiple benchmark multi-view and multi-label data sets verify the effectiveness of TM3L compared with the state-of-the-art solutions.

PVRAR: Point-View Relation Neural Network Embedded with Both Attention Mechanism and Radon Transform for 3D Shape Recognition

Owing to the favorable performance of deep neural networks for 3D shape recognition, an increasing number of researchers are interested in designing novel 3D shape descriptors. However, the relationship between multiple views and point clouds needs to be further elucidated. We propose a multimodal method that combines the features of point clouds and multiple views, i.e., point-view relation neural network embedded with both attention mechanism and Radon transform, to obtain better descriptors. First, a two-dimensional linear Radon transform is performed to investigate linear and color features in multiple views, and the features are used as the input of our network to enable significant distinctions between different views. Moreover, a convolutional block attention module is adopted to enhance the features of point clouds and hence improve the expression ability of feature descriptors. The effectiveness of the proposed method is verified using ModelNet40 and ModelNet10 datasets. Experimental results show that our method can effectively improve the capability of feature extraction and expression as well as achieve state-of-the-art performance on two well-known 3D datasets.

Continual Multiview Task Learning via Deep Matrix Factorization.

The state-of-the-art multitask multiview (MTMV) learning tackles a scenario where multiple tasks are related to each other via multiple shared feature views. However, in many real-world scenarios where a sequence of the multiview task comes, the higher storage requirement and computational cost of retraining previous tasks with MTMV models have presented a formidable challenge for this lifelong learning scenario. To address this challenge, in this article, we propose a new continual multiview task learning model that integrates deep matrix factorization and sparse subspace learning in a unified framework, which is termed deep continual multiview task learning (DCMvTL). More specifically, as a new multiview task arrives, DCMvTL first adopts a deep matrix factorization technique to capture hidden and hierarchical representations for this new coming multiview task while accumulating the fresh multiview knowledge in a layerwise manner. Then, a sparse subspace learning model is employed for the extracted factors at each layer and further reveals cross-view correlations via a self-expressive constraint. For model optimization, we derive a general multiview learning formulation when a new multiview task comes and apply an alternating minimization strategy to achieve lifelong learning. Extensive experiments on benchmark data sets demonstrate the effectiveness of our proposed DCMvTL model compared with the existing state-of-the-art MTMV and lifelong multiview task learning models.

Hierarchical Graph Attention Based Multi-View Convolutional Neural Network for 3D Object Recognition

For multi-view convolutional neural network based 3D object recognition, how to fuse the information of multiple views is a key factor affecting the recognition performance. Most traditional methods use max-pooling algorithm to obtain the final 3D object feature, which does not take into account the correlative information between different views. To make full use of the effective information of multiple views, this paper introduces the hierarchical graph attention based multi-view convolutional neural network for 3D object recognition. At first, the view selection module is proposed to reduce redundant view information in multiple views, which can select the projective views with more effective information. Then, the correlation weighted feature aggregation module is proposed to better fuse multiple view features. Finally, the hierarchical feature aggregation network structure is designed to further to make full use of the correlation information of multiple views. Extensive experimental results have validated the effectiveness of the proposed method.

Multi-View Multi-Label Learning With Sparse Feature Selection for Image Annotation

In image analysis, image samples are always represented by multiple view features and associated with multiple class labels for better interpretation. However, multiple view data may include noisy, irrelevant and redundant features, while multiple class labels can be noisy and incomplete. Due to the special data characteristic, it is hard to perform feature selection on multi-view multi-label data. To address these challenges, in this paper, we propose a novel multi-view multi-label sparse feature selection (MSFS) method, which exploits both view relations and label correlations to select discriminative features for further learning. Specifically, the multi-labeled information is decomposed into a reduced latent label representation to capture higher level concepts and correlations among multiple labels. Multiple local geometric structures are constructed to exploit visual similarities and relations for different views. By taking full advantage of the latent label representation and multiple local geometric structures, the sparse regression model with an $l_{2,1}$ -norm and an Frobenius norm ( $F$ -norm) penalty terms is utilized to perform hierarchical feature selection, where the $F$ -norm penalty performs high-level (i.e., view-wise) feature selection to preserve the informative views and the $l_{2,1}$ -norm penalty conducts low-level (i.e., row-wise) feature selection to remove noisy features. To solve the proposed formulation, we also devise a simple yet efficient iterative algorithm. Experiments and comparisons on real-world image datasets demonstrate the effectiveness and potential of MSFS.

Multi-view clustering via neighbor domain correlation learning

With the development of data science, more and more data are presented in the form of multi-view. Compared with single-view feature learning, multi-view feature learning is more effective, and it has been successfully applied in many fields. Clustering is a core technology of computer science. Thus, many researchers start to study multi-view clustering. Recently, combining with multi-view feature learning techniques, some multi-view clustering methods have been presented. These methods mainly focus on the multiple features fusion, while most of them ignore the correlations among multiple views. Therefore, it cannot make full use of the advantages of multiple view features. In this paper, we propose a novel approach, named multi-view clustering via neighbor domain correlation learning (MCNDCL) approach. Specifically, MCNDCL learns a discriminant common space for multiple view features. Under the learned common space, the correlations of the consistent neighbor domain are maximized, and the correlations of specific neighbor domain are minimized at the same time. Extensive experimental results on four typical benchmarks, i.e., UCI Digits, Caltech7, BBCSport and CCV, validate the high effectiveness of our proposed approach.