Multiple Subspaces Research Articles

FACapsnet: A fusion capsule network with congruent attention for cyberbullying detection

As an extension of bullying on social networks, cyberbullying has seriously affected the security of the online social environment and infringed on mental health. Taking appropriate measures to detect online bullying reviews is crucial. Existing studies usually classify the whole content as cyberbullying and non-cyberbullying. However, they do not fully exploit the interaction of multi-dimensional features and precisely distinguish the types of cyberbullying. To automatically extract features of bullying words and further identify fine-grained types of cyberbullying better, we propose a fusion capsule network with congruent attention for cyberbullying detection. In the proposed algorithm, a novel similarity weighting scheme based on word2vec is designed to soft highlight bullying features in word embeddings. Meanwhile, to leverage the respective advantages of extracted multiple subspace features, we construct a novel extensible congruent attention to balance the fusion of complex correlations between different subspace representations and retain the independence of context features. The fused features are updated iteratively with dynamic routing to aggregate and generate fine-grained category capsules for cyberbullying prediction. A series of experiments on the tweets cyberbullying benchmark demonstrate that our architecture matches or exceeds the performance of the compared baseline models and the results of extensive experiments prove the effectiveness of different strategies.

A Progressive Plane Detection Filtering Method for Airborne LiDAR Data in Forested Landscapes

Ground filtering is necessary in processing airborne light detection and ranging (LiDAR) point clouds for forestry applications. This study proposes a progressive plane detection filtering (PPDF) method. First, the method uses multi-scale planes to characterize terrain, i.e., the local terrain with large slope variations is represented by small-scale planes, and vice versa. The planes are detected in local point clouds by the random sample consensus method with decreasing plane sizes. The reliability of the planes to represent local terrain is evaluated and the planes with optimal sizes are selected according to evaluation results. Then, ground seeds are identified by selecting the interior points of the planes. Finally, ground points are iteratively extracted based on the reference terrain, which is constructed using evenly distributed neighbor ground points. These neighbor points are identified by selecting the nearest neighbor points of multiple subspaces, which are divided from the local space with an unclassified point as center point. PPDF was tested in six sites with various terrain and vegetation characteristics. Results showed that PPDF was more accurate and robust compared to the classic filtering methods including maximum slope, progressive morphology, cloth simulation, and progressive triangulated irregular network densification filtering methods, with the smallest average total error and standard deviation of 3.42% and 2.45% across all sites. Moreover, the sensitivity of PPDF to parameters was low and these parameters can be set as fixed values. Therefore, PPDF is effective and easy-to-use for filtering airborne LiDAR data.

ConvMS: Improving Convolutional Knowledge Graph Embeddings via Integrating Information of Multiple Subspaces

Knowledge graphs are involved in more and more applications to further improve intelligence. Owing to the inherent incompleteness of knowledge graphs resulted from data updating and missing, a number of knowledge graph completion models are proposed in succession. To obtain better performance, many methods are of high complexity, making it time-consuming for training and inference. This paper proposes a simple but effective model using only shallow neural networks, which combines enhanced feature interaction and multi-subspace information integration. In the enhanced feature interaction module, entity and relation embeddings are almost peer-to-peer interacted via multi-channel 2D convolution. In the multi-subspace information integration module, entity and relation embeddings are projected to multiple subspaces to extract multi-view information to further boost performance. Extensive experiments on widely used datasets show that the proposed model outperforms a series of strong baselines. And ablation studies demonstrate the effectiveness of each submodule in the model.

Pursuit of a discriminative representation for multiple subspaces via sequential games

We consider the problem of learning discriminative representations for data in a high-dimensional space with distribution supported on or around multiple low-dimensional linear subspaces. That is, we wish to compute a linear injective map of the data such that the features lie on multiple orthogonal subspaces. Instead of solving this learning problem using multiple instances of principal component analysis (PCA), we cast it as a sequential game using the closed-loop transcription (CTRL) framework recently proposed for learning discriminative and generative representations for general low-dimensional submanifolds. We prove that the equilibrium solutions to the game indeed give the correct representations. Our approach unifies classical methods of learning subspaces with modern machine learning practice, by showing that subspace learning problems may be provably solved using the modern toolkit of representation learning. In addition, our work provides the first theoretical justification for the CTRL framework in the important special case of linear subspaces. We support our theoretical findings with compelling empirical evidence. We also generalize the sequential game formulation to more general representation learning problems. Our code is publicly available on GitHub.

A Covariate-Regulated Sparse Subspace Learning Model and Its Application to Process Monitoring and Fault Isolation

Multivariate functional data are increasingly common in various applications. The cross-correlation of different process variables is typically complex in that a variable might be weakly correlated or not correlated with most of the other variables, and the cross-correlation is time-varying and might be regulated by some exogenous covariates. To address these two challenges, we propose a covariate-regulated sparse subspace learning (CSSL) model. We consider the scenario that these process variables lie in multiple subspaces, and only process variables from the same subspace are cross-correlated with each other. To take into account the effect of the exogenous covariates on the subspace structure, we partition the domain of the covariates into a number of regions. In each region, the subspace structure is treated as constant and can be learned independently. An efficient decision-tree-based algorithm is then proposed to obtain the solution. The proposed method can be further applied to process monitoring and fault isolation for multivariate processes. The efficacy of this method is demonstrated by comprehensive simulations and a case study on a dataset from the supervisory control and data acquisition (SCADA) system of the wind turbine.

Anomaly Detection for Hyperspectral Imagery via Tensor Low-Rank Approximation With Multiple Subspace Learning

Hyperspectral anomaly detection (HAD) is regarded as an indispensable, pivotal technology in remote sensing and earth science domains. Nevertheless, most existing detection approaches for anomaly targets flatten 3-D hyperspectral images (HSIs) with spatial and spectral information into 2-D spectral vector data, which virtually breaks up the internal spatial structure in HSIs and degenerates the detection performance. To this end, we directly consider the HSI data cube as a 3-D tensor and develop a novel tensor low-rank approximation (TLRA) detection algorithm to separate the sparse anomalous component from the background with low-rank characteristics. Then, in light of the multi-subspace structure in heterogeneous backgrounds, we utilize multiple subspace learning (MSL) theory to encode the background tensor with a coefficient tensor and corresponding dictionary tensor. In addition, considering that different singular values indicate different information quantities and should be penalized to different extents, we introduce a tighter tensor rank surrogate named the ϵ-shrinkage tensor nuclear norm (ϵ-TNN) to recover the low-rank component more accurately. Meanwhile, concerning the sparse anomaly target, the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> constraint is incorporated to represent the group sparsity of the abnormal component. Finally, an effective iterative optimization algorithm based on the alternating direction method of multipliers (ADMM) is devised to solve the proposed TLRA-MSL model. We conduct extensive experiments on six hyperspectral datasets to prove the effectiveness and robustness of our method. The experimental results illustrate that better detection performance is obtained using the proposed model compared with other state-of-the-art algorithms.

Learning Tensor Low-Rank Representation for Hyperspectral Anomaly Detection.

Recently, low-rank representation (LRR) methods have been widely applied for hyperspectral anomaly detection, due to their potentials in separating the backgrounds and anomalies. However, existing LRR models generally convert 3-D hyperspectral images (HSIs) into 2-D matrices, inevitably leading to the destruction of intrinsic 3-D structure properties in HSIs. To this end, we propose a novel tensor low-rank and sparse representation (TLRSR) method for hyperspectral anomaly detection. A 3-D TLR model is expanded to separate the LR background part represented by a tensorial background dictionary and corresponding coefficients. This representation characterizes the multiple subspace property of the complex LR background. Based on the weighted tensor nuclear norm and the LF,1 sparse norm, a dictionary is designed to make its atoms more relevant to the background. Moreover, a principal component analysis (PCA) method can be assigned as one preprocessing step to exact a subset of HSI bands, retaining enough the HSI object information and reducing computational time of the postprocessing tensorial operations. The proposed model is efficiently solved by the well-designed alternating direction method of multipliers (ADMMs). A comparison with the existing algorithms via experiments establishes the competitiveness of the proposed method with the state-of-the-art competitors in the hyperspectral anomaly detection task.

Multiple discriminant preserving support subspace RBFNNs with graph similarity learning

In a high-dimensional sample space, the distribution characteristics of samples are complex. It is difficult to accurately describe the local distribution characteristics of samples directly in the original sample space. This paper proposes a multiple discriminant preserving support subspace radial basis function neural networks with graph similarity learning (DPSS-RBFNN) to describe the distribution characteristics of samples in the original high-dimensional space by multiple low-dimensional and simple discriminant preserving subspaces. DPSS-RBFNN includes the discriminant preserving support subspace (DPSS) learning module and the subspace distribution feature extraction (SDFE) module. In the DPSS learning module, the discriminativeness of each attribute and the joint discriminant between attributes are first considered to construct multiple subspaces. The discriminativeness of the features in these subspaces is not lower than that of the original samples. The graph similarity learning is used to measure the similarity of sample distributions between subspaces with different dimensions. Then multiple DPSSs are obtained. In SDFE module, the distribution characteristics of the samples are described by combining the local responses of the feature space extracted by the sub-RBFNN in each DPSS. The experimental results show that the proposed DPSS-RBFNN with few kernels can achieve higher accuracy in the recognition task than state-of-the-art algorithms.

Automatic Target Recognition Based on RCS and Angular Diversity for Multistatic Passive Radar

Asan important means in low-altitude airspace surveillance, passive radar has been widely studied. With the increasing complexity of the low-altitude environment, the urgent need for radar's capability of automatic target recognition (ATR) emerges. In the field of ATR for passive radar, the method based on radar cross section (RCS) is an important direction for the simplicity of feature extraction. In this article, we exploit the compensated quasi-echo-power (CQEP), i.e., quasi-RCS, to realize ATR in passive radar. We propose an ATR method on the basis of the angular diversity. First, we divide the angular space, which consists of four components of the incident angle and scattering angle of electromagnetic wave, into multiple subspaces. Then, we build a subrecognizer in each angular subspace according to the CQEP distribution characteristics. Each receiving station of the multistatic passive radar recognizes targets in an angular subspace corresponding to its observation and outputs a preliminary decision. Based on the preliminary decisions of all receiving stations, a voting strategy is adopted to make the final decision on the unknown target class in the fusion center. Experimental results based on real-life target data show that the proposed method has a significant improvement on the average correct recognition rate with fusion of three receiving stations, which validates the proposed method.

SC-GAN: Subspace Clustering based GAN for Automatic Expression Manipulation

In recent years, the topics of facial attribute manipulation and decomposition have gained great popularity in computer vision and human computer interaction. Even though such methods have been preliminarily employed in some photo beautification applications, it still remains challenging due to the highly-versatile facial attributes and their drastic appearance changes subject to variation of deformation, illumination, pose, etc. The prevailing problems are especially severe when we are faced with group photos involving many faces. To overcome such critical limitations and discover more meaningful visual attributes and their possible decompositions, we develop a subspace clustering based generative adversarial network (SC-GAN) in this paper. Our SC-GAN can simultaneously decompose multiple subspaces and generate diverse samples correspondingly, thus the training of the generative models could be more effectively guided by facial attribute and its decomposition and manipulation in a natural and meaningful fashion. Our SC-GAN incorporates the SIFT K-means cluster, which could split the holistic semantic facial space into different subspaces without supervision, and help the new GAN generate more convincing results within specific subspaces. Extensive experiments and comprehensive evaluations confirm that, our method can greatly reduce the unexpected influences caused by portrait diversities and outperform the state-of-the-art facial attribute manipulation approaches.11https://github.com/buaaswf/SC-GAN/

Multi-embedding space set-kernel and its application to multi-instance learning

Set-level problems become critical when we are interested in animals in pictures, links in web pages, and components in drugs. The key issue is to measure the similarity between two sets. This paper develops a data-dependent multi-embedding space set-kernel (MSK) with close to linear time complexity and applies it to multi-instance learning (MIL), which is a typical set-level problem. The majority of current set-kernels are independent of the underlying data distribution. In contrast, MSK indirectly measures set similarity by determining the relationship between embedding vectors. Each set’s embedding vectors are new representations with controlled dimensionality in the multi-embedding space. Multi-embedding space is described here as a set containing multiple subspaces based on the distribution of the data set. In addition, the MSK feature map is used to speed up the computation of similarity over the entire data set. Extensive experiments were done on 46 MIL data sets across five application domains. The results demonstrate that MSK has the lowest average classification loss and the highest stability compared with the rival set-kernels. The linear time complexity is also verified. Source codes are available athttps://github.com/InkiInki/MSK.

Comprehensive Multi-view Representation Learning

Recently, Multi-view Representation Learning (MRL) has drawn immense attentions in the analysis of multi-source data and ubiquitously employed across different research fields. This important issue is designed to learn a feature representation with sufficient information from multiple views. In this paper, we propose a novel Comprehensive Multi-view Representation Learning (CMRL), which can fully explore available information contained in both the feature representations and subspace representations of multiple views. The desired feature representation learned in CMRL profits from the consistency and complementarity of multi-view data. Specifically, the complementary information is mined by applying the degeneration mapping model on multiple feature representations, the consensus information is explored by imposing a low-rank tensor constraint on multiple subspace representations. Further, the objective function of CMRL is optimized by an Augmented Lagrangian Multiplier (ALM) based algorithm. Finally, our CMRL is evaluated on seven benchmark multi-view datasets and compared with several state-of-the-art methods, experimental results illustrate the superiority and effectiveness of the proposed method. What is more, we find that the proposed method can also be successfully applied to multi-view subspace clustering and achieves promising clustering results.

The Robust Multi-Scale Deep-SVDD Model for Anomaly Online Detection of Rolling Bearings.

Aiming at the online detection problem of rolling bearings, the limited amount of target bearing data leads to insufficient model in training and feature representation. It is difficult for the online detection model to construct an accurate decision boundary. To solve the problem, a multi-scale robust anomaly detection method based on data enhancement technology is proposed in this paper. Firstly, the training data are transformed into multiple subspaces through the data enhancement technology. Then, a prototype clustering method is introduced to enhance the robustness of features representation under the framework of the robust deep auto-encoding algorithm. Finally, the robust multi-scale Deep-SVDD hyper sphere model is constructed to achieve online detection of abnormal state data. Experiments are conducted on the IEEE PHM Challenge 2012 bearing data set and XJTU-TU data set. The proposed method shows much greater susceptibility to incipient faults, and it has fewer false alarms. The robust multi-scale Deep-SVDD hyper sphere model significantly improves the performance of incipient fault detection for rolling bearings.

SPCBPT

Bidirectional path tracing (BDPT) can be accelerated by selecting appropriate light sub-paths for connection. However, existing algorithms need to perform frequent distribution reconstruction and have expensive overhead. We present a novel approach, SPCBPT, for probabilistic connections that constructs the light selection distribution in sub-path space. Our approach bins the sub-paths into multiple subspaces and keeps the sub-paths in the same subspace of low discrepancy, wherein the light sub-paths can be selected by a subspace-based two-stage sampling method, i.e., first sampling the light subspace and then resampling the light sub-paths within this subspace. The subspace-based distribution is free of reconstruction and provides efficient light selection at a very low cost. We also propose a method that considers the Multiple Importance Sampling (MIS) term in the light selection and thus obtain an MIS-aware distribution that can minimize the upper bound of variance of the combined estimator. Prior methods typically omit this MIS weights term. We evaluate our algorithm using various benchmarks, and the results show that our approach has superior performance and can significantly reduce the noise compared with the state-of-the-art method.

Parallelizing MCMC sampling via space partitioning

Efficient sampling of many-dimensional and multimodal density functions is a task of great interest in many research fields. We describe an algorithm that allows parallelizing inherently serial Markov chain Monte Carlo (MCMC) sampling by partitioning the space of the function parameters into multiple subspaces and sampling each of them independently. The samples of the different subspaces are then reweighted by their integral values and stitched back together. This approach allows reducing sampling wall-clock time by parallel operation. It also improves sampling of multimodal target densities and results in less correlated samples. Finally, the approach yields an estimate of the integral of the target density function.

RASCL: a randomised approach to subspace clusters

Subspace clustering aims to discover clusters in projections of highly dimensional numerical data. In this paper, we focus on discovering small collections of highly interesting subspace clusters that do not try to cluster all data points, leaving noisy data points unclustered. To this end, we propose a randomised method that first converts the highly dimensional database to a binarised one using projected samples of the original database. Subsequently, this database is mined for frequent itemsets, which we show can be translated back to subspace clusters. In this way, we are able to explore multiple subspaces of different sizes at the same time. In our extensive experimental analysis, we show on synthetic as well as real-world data that our method is capable of discovering highly interesting subspace clusters efficiently.

Robust Latent Multi-Source Adaptation for Encephalogram-Based Emotion Recognition.

In practical encephalogram (EEG)-based machine learning, different subjects can be represented by many different EEG patterns, which would, in some extent, degrade the performance of extant subject-independent classifiers obtained from cross-subjects datasets. To this end, in this paper, we present a robust Latent Multi-source Adaptation (LMA) framework for cross-subject/dataset emotion recognition with EEG signals by uncovering multiple domain-invariant latent subspaces. Specifically, by jointly aligning the statistical and semantic distribution discrepancies between each source and target pair, multiple domain-invariant classifiers can be trained collaboratively in a unified framework. This framework can fully utilize the correlated knowledge among multiple sources with a novel low-rank regularization term. Comprehensive experiments on DEAP and SEED datasets demonstrate the superior or comparable performance of LMA with the state of the art in the EEG-based emotion recognition.

Feedback exponential stabilization of stochastic quantum systems based on state space division

In this paper, the global exponential stabilization of two-level stochastic quantum systems is achieved by using measurement-based feedback strategies, including state feedback and noise-assisted feedback, and the state convergence rate is improved by dividing the state space into multiple subspaces and using different feedback controls in different state subspaces. For both the combination of two continuous noise-assisted feedback and the combination of state feedback and noise-assisted feedback, the state space is divided based on the comparison of real-time state convergence rate under different measurement-based feedback strategies, respectively. The global exponential convergence and the effect of dividing state space in improving state convergence rate for two-level quantum systems are proved in theory and verified in numerical simulations.

Multi-view Subspace Clustering with View Correlations via low-rank tensor learning

With the purpose of boosting clustering performance, the manner of excavating underlying view correlations is an important issue of multi-view subspace clustering (MSC). Nevertheless, regarding most MSC approaches are centered on the view correlations in multiple subspace representations and neglect the view correlations in multiple feature representations. To address this limitation, this paper introduces a method, dubbed Multi-view Subspace Clustering with View Correlations (MSCVC), which excavates underlying view correlations in both multiple feature representations and multiple subspace representations simultaneously. Specifically, the Multi-view Principal Component Analysis (MPCA) is designed to capture the view correlations contained in multiple feature representations by using the orthogonal mapping and low-rank tensor constraint. As a consequence, view correlations in multiple feature representations can be effectively investigated and simultaneously embedded in the learned new feature representations. For view correlations in multiple subspace representations, the new feature representations learned in MPCA are employed and the low-rank tensor constraint is used as well. Experiments are conducted on nine benchmark datasets to demonstrate the progressiveness of our MSCVC compared to several state-of-the-art algorithms.

Parallel Cooperative Coevolutionary Grey Wolf Optimizer for Path Planning Problem of Unmanned Aerial Vehicles.

The path planning of Unmanned Aerial Vehicles (UAVs) is a complex and hard task that can be formulated as a Large-Scale Global Optimization (LSGO) problem. A higher partition of the flight environment leads to an increase in route’s accuracy but at the expense of greater planning complexity. In this paper, a new Parallel Cooperative Coevolutionary Grey Wolf Optimizer (PCCGWO) is proposed to solve such a planning problem. The proposed PCCGWO metaheuristic applies cooperative coevolutionary concepts to ensure an efficient partition of the original search space into multiple sub-spaces with reduced dimensions. The decomposition of the decision variables vector into several sub-components is achieved and multi-swarms are created from the initial population. Each sub-swarm is then assigned to optimize a part of the LSGO problem. To form the complete solution, the representatives from each sub-swarm are combined. To reduce the computation time, an efficient parallel master-slave model is introduced in the proposed parameters-free PCCGWO. The master will be responsible for decomposing the original problem and constructing the context vector which contains the complete solution. Each slave is designed to evolve a sub-component and will send the best individual as its representative to the master after each evolutionary cycle. Demonstrative results show the effectiveness and superiority of the proposed PCCGWO-based planning technique in terms of several metrics of performance and nonparametric statistical analyses. These results show that the increase in the number of slaves leads to a more efficient result as well as a further improved computational time.