Individual Subspaces Research Articles

Adaptive Multi-Model Predictive Control with Optimal Model Bank Formation: Consideration of Local Models Uncertainty and Stability

The Multiple Model Control (MMC) structure comprises three main components: the model bank, controller bank, and supervisor algorithm. Precise design of these components is crucial for achieving high control performance within the MMC framework, albeit this effort is not without its challenges. These challenges involve optimizing the model and controller banks ensuring system stability when dealing with uncertainties in the local models and enabling smooth switching between model-controller pairs. This paper addresses these challenges by presenting a comprehensive approach. Firstly, the optimal model bank is designed using an automatic clustering approach. Then, the design of the adaptive multi-model predictive controller bank and a supervisor algorithm capable of performing soft switching are discussed. The proposed control system exhibits the capability to ensure closed-loop system stability within each individual subspace, as well as during the transitioning between distinct subspaces. This stability is preserved even in the face of inherent uncertainties associated with the local models comprising the model bank. To evaluate and validate the performance of the proposed control system, it is applied to a satellite attitude control system. The results confirm the effectiveness and performance of the control system. The proposed control system holds promise for controlling highly nonlinear, complex, or switched systems, ensuring closed-loop system stability, and achieving high control performance.

Wavelet Cross-Correlation Signal Processing for Two-Phase Flow Control System in Oil Well Production

An algorithm based on continuous measurement of multiphase flows of oil well production has been designed to improve the efficiency of the technical control of oil production processes in the field. Separation-free, non-contact measurement of multiphase flows of oil well products allows increasing the efficiency of managing oil production processes in the field. Monitoring the current density using radioisotope measuring transducers (RMTs) allows obtaining information about the structure of the flow in the form of the distribution of gas inclusions and the speed of movement of liquid and gas in a two-phase flow. Fluid velocity measurement is based on digital processing of RMT signals, applying a continuous or discrete undecimated wavelet transform to them, and assessing the cross-correlation of wavelet coefficients in individual subspaces of the wavelet decomposition. The cross-correlation coefficients of two RMT signals located at a base distance, calculated in the subspaces of the wavelet decomposition, characterize the speed of movement of gas bubbles of different sizes in a vertical pipe. The measurement assumes that the velocity of the liquid phase of the oil flow in a vertical pipe mainly corresponds to the velocity of small bubbles. This speed should be determined by the maximum cross-correlation of wavelet coefficients in the corresponding decomposition subspace. Computer modeling made it possible to evaluate the characteristics of the algorithm for controlling the speed of liquid movement in the gas–liquid flow of oil well products and determine the mass flow rate of the liquid and the relative value of the gas content. The implementation of the algorithm in a multi-channel version of the device allows monitoring an entire cluster of wells in the field.

Epistasis and pleiotropy shape biophysical protein subspaces associated with drug resistance.

Protein space is a rich analogy for genotype-phenotype maps, where amino acid sequence is organized into a high-dimensional space that highlights the connectivity between protein variants. It is a useful abstraction for understanding the process of evolution, and for efforts to engineer proteins towards desirable phenotypes. Few mentions of protein space consider how protein phenotypes can be described in terms of their biophysical components, nor do they rigorously interrogate how forces like epistasis-describing the nonlinear interaction between mutations and their phenotypic consequences-manifest across these components. In this study, we deconstruct a low-dimensional protein space of a bacterial enzyme (dihydrofolate reductase; DHFR) into "subspaces" corresponding to a set of kinetic and thermodynamic traits [k_{cat},K_{M},K_{i}, and T_{m} (melting temperature)]. We then examine how combinations of three mutations (eight alleles in total) display pleiotropy, or unique effects on individual subspace traits. We examine protein spaces across three orthologous DHFR enzymes (Escherichia coli, Listeria grayi, and Chlamydia muridarum), adding a genotypic context dimension through which epistasis occurs across subspaces. In doing so, we reveal that protein space is a deceptively complex notion, and that future applications to bioengineering should consider how interactions between amino acid substitutions manifest across different phenotypic subspaces.

Robust semi-supervised multi-view graph learning with sharable and individual structure

The construction of a high-quality multi-view consensus graph is key to graph-based semi-supervised multi-view learning (GSSMvL) methods. However, most existing GSSMvL methods explore sample relationships in the original multi-view feature space, which obtains a contaminated graph that cannot reveal the underlying manifold structure of the samples. Moreover, traditional GSSMvL methods fail to explore the diverse structures of multi-view features, which may lose their complementary information and lead to a suboptimal graph. In this paper, we propose a novel unified robust semi-supervised multi-view graph learning framework based on the sharable and individual structure (RSSMvSI), which can eliminate the influence of noise and exploit the knowledge of multi-view data in a reasonable manner. Specifically, we first learn clean data by manipulating sparse noise with l2,1 norm. We then simultaneously explore the sharable and individual self-representation subspace on the learned clean multi-view data. The key point is that noisy data does not participate in subspace learning, which improves the robustness of the proposed method. By constructing the optimal consensus graph with the learned sharable and individual subspace, RSSMvSI can better utilize the complementary information of multi-view data and approximate the manifold structure of samples. To the best of our knowledge, this is the first attempt to learn the self-representation subspace on recovered multi-view clean data. Extensive experiments on various real-world multi-view datasets demonstrate the superiority and robustness against state-of-the-art methods.

Discovering Community Structure in Multiplex Networks via a Co-Regularized Robust Tensor-Based Spectral Approach

Complex networks arise in various fields, such as biology, sociology and communication, to model interactions among entities. Entities in many real-world systems exhibit different types of interactions, which requires modeling these type of systems properly. Multiplex networks are used to model these systems, as they can reflect the nodes’ pair-wise interactions as multiple distinct types of links across layers. Community detection is a widely studied application in network analysis as it provides insights into the structure and organization of the network. Even though multiple algorithms have been developed in the community detection field, many of them have a limited performance in the presence of noise. In this article, we develop a novel algorithm that combines tensor low-rank representation, spectral clustering and distance regularization to improve the accuracy in discovering communities in multiplex networks. The low-rank representation leads to reducing the noise and errors existing in the network and the optimization of an accurate consensus set of eigenvectors that reveals the communities in the network. Moreover, the proposed approach balances the agreement between the eigenvectors of each layer, i.e., individual subspaces, and the consensus set of eigenvectors, i.e., common subspaces, by minimizing the projection distance between them. The common and individual subspaces are computed efficiently through Tucker decomposition and modified spectral clustering, respectively. Finally, multiple experiments are conducted on real and simulated networks to evaluate the proposed approach and compare it to state-of-the-art algorithms. The proposed approach shows its robustness and efficiency in discovering the communities in multiplex networks.

Discrete matrix factorization cross-modal hashing with multi-similarity consistency

AbstractRecently, matrix factorization-based hashing has gained wide attention because of its strong subspace learning ability and high search efficiency. However, some problems need to be further addressed. First, uniform hash codes can be generated by collective matrix factorization, but they often cause serious loss, degrading the quality of hash codes. Second, most of them preserve the absolute similarity simply in hash codes, failing to capture the inherent semantic affinity among training data. To overcome these obstacles, we propose a Discrete Multi-similarity Consistent Matrix Factorization Hashing (DMCMFH). Specifically, an individual subspace is first learned by matrix factorization and multi-similarity consistency for each modality. Then, the subspaces are aligned by a shared semantic space to generate homogenous hash codes. Finally, an iterative-based discrete optimization scheme is presented to reduce the quantization loss. We conduct quantitative experiments on three datasets, MSCOCO, Mirflickr25K and NUS-WIDE. Compared with supervised baseline methods, DMCMFH achieves increases of $$0.22\%$$ 0.22 % , $$3.00\%$$ 3.00 % and $$0.79\%$$ 0.79 % on the image-query-text tasks for three datasets respectively, and achieves increases of $$0.21\%$$ 0.21 % , $$1.62\%$$ 1.62 % and $$0.50\%$$ 0.50 % on the text-query-image tasks for three datasets respectively.

Adaptive risk-aware sharable and individual subspace learning for cancer survival analysis with multi-modality data.

Biomedical multi-modality data (also named multi-omics data) refer to data that span different types and derive from multiple sources in clinical practices (e.g. gene sequences, proteomics and histopathological images), which can provide comprehensive perspectives for cancers and generally improve the performance of survival models. However, the performance improvement of multi-modality survival models may be hindered by two key issues as follows: (1) how to learn and fuse modality-sharable and modality-individual representations from multi-modality data; (2) how to explore the potential risk-aware characteristics in each risk subgroup, which is beneficial to risk stratification and prognosis evaluation. Additionally, learning-based survival models generally refer to numerous hyper-parameters, which requires time-consuming parameter setting and might result in a suboptimal solution. In this paper, we propose an adaptive risk-aware sharable and individual subspace learning method for cancer survival analysis. The proposed method jointly learns sharable and individual subspaces from multi-modality data, whereas two auxiliary terms (i.e. intra-modality complementarity and inter-modality incoherence) are developed to preserve the complementary and distinctive properties of each modality. Moreover, it equips with a grouping co-expression constraint for obtaining risk-aware representation and preserving local consistency. Furthermore, an adaptive-weighted strategy is employed to efficiently estimate crucial parameters during the training stage. Experimental results on three public datasets demonstrate the superiority of our proposed model.

Formula omitted]: A logically randomized forest algorithm for classification and regression problems

Tree-based ensemble algorithms (TEAs) have made significant advances in recent years due to their simple algorithmic design. However, when the proportion of the ‘most informative’ features is low, the performance of conventional TEAs degrades significantly. The primary rationale for performance degradation is that traditional algorithmic design appears to be biased toward the least informative features, and the sub-space selection procedure contains uninformative features. This paper proposes a logically randomized forest (LRF) algorithm by incorporating two different enhancements into existing TEAs. The first enhancement is made to address the issue of biasness by performing feature-level engineering. The second enhancement is the approach by which individual feature sub-spaces are selected. To derive the first enhancement, we use the graph-theoretic principle of minimal vertex cover to construct a relevant assemblage of features. Following that, the permutation-based feature importance technique is employed to calculate the ‘informativeness’ of the relevant features in order to infuse logical randomness into the individual trees in the forest. For the second enhancement, the stratified sampling method is used to ensure that the most informative features are present in all newly created feature sub-spaces. Consequently, individual trees are generated using the Roulette wheel-based selection (RWS) algorithm. The proposed algorithm has been evaluated on two real-world genomic data sets, ten hybrid-synthetic classification data sets, and twenty multidisciplinary benchmark data sets with varying characteristics. The experimental findings demonstrate that the LRF outperforms the existing benchmark and cutting-edge TEAs.

3DPointCaps++: Learning 3D Representations with Capsule Networks

We present 3DPointCaps++ for learning robust, flexible and generalizable 3D object representations without requiring heavy annotation efforts or supervision. Unlike conventional 3D generative models, our algorithm aims for building a structured latent space where certain factors of shape variations, such as object parts, can be disentangled into independent sub-spaces. Our novel decoder then acts on these individual latent sub-spaces (i.e. capsules) using deconvolution operators to reconstruct 3D points in a self-supervised manner. We further introduce a cluster loss ensuring that the points reconstructed by a single capsule remain local and do not spread across the object uncontrollably. These contributions allow our network to tackle the challenging tasks of part segmentation, part interpolation/replacement as well as correspondence estimation across rigid / non-rigid shape, and across / within category. Our extensive evaluations on ShapeNet objects and human scans demonstrate that our network can learn generic representations that are robust and useful in many applications.

Subspace cross representation measure for robust face recognition with few samples

Similarity measure generally exerts a crucial role in face recognition. Recently, regression analysis based similarity measure mechanism has demonstrated significant potential in robust face recognition. Nevertheless, most existing regression methods are far from perfect under few samples due to the poor performance of spanning the individual subspace. Previous works have been noticed that the singular value decomposition (SVD) of facial image can generate a set of complete base of individual subspace. Then we present a novel and efficient image similarity measure model named subspace cross representation (SCR) measure for face recognition with few samples. The power of our proposed SCR stems from the following facts. One is that the complete base can weaken the dependence of linear regression method on the number of labeled samples. The other is the cross linear representation can effectively use two-dimensional geometric features generated by SVD to distinguish facial images. The validity of SCR is tested by a large amount of experiments on AR, CUHK Sketch, Extended Yale B databases, etc. The experimental results demonstrate that SCR achieves satisfactory recognition accuracy compared with other methods, under few sample condition.

A Novel Algorithm Based on a Common Subspace Fusion for Visual Object Tracking

Recent methods for visual tracking exploit a multitude of information obtained from combinations of handcrafted and/or deep features. However, the response maps derived from these feature combinations are often fused using simple strategies such as winner-takes-all or weighted sum approaches. Although some efficient fusion methods have also been proposed, these methods still do not leverage the individual strengths of the different features being fused. In the current work, we propose a novel information fusion strategy comprising a common low-rank subspace for the fusion of different types of features and tracker responses. Firstly, we interpret the response maps as smoothly varying functions which can be efficiently represented using individual low-rank matrices, thus removing high frequency noise and sparse artifacts. Secondly, we estimate a common low-rank subspace which is constrained to remain close to each individual low-rank subspace resulting in an efficient fusion strategy. The proposed algorithm achieves good performance by integrating the information contained in heterogeneous feature types. We demonstrate the efficiency of our algorithm using several combinations of features as well as correlation filter and end-to-end deep trackers. The proposed common subspace fusion algorithm is generic and can be used to efficiently fuse the response maps of varying types of feature representations as well as trackers. Extensive experiments on several tracking benchmarks including OTB100, TC128, VOT-ST 2018, VOT-LT 2018, UAV123, GOT-10K and LaSoT have demonstrated significant performance improvements compared to many SOTA tracking methods.

High-Resolution SAR Image Classification Using Subspace Wavelet Encoding Network

The feature learning methods based on convolutional neural networks (CNNs) have produced tremendous achievements in high-resolution (HR) synthetic aperture radar (SAR) image classification. However, the inherent speckle noise could weaken the effectiveness of the convolutional feature statistics. To effectively characterize the features of SAR land-covers under speckle noise, we propose a subspace wavelet encoding network (SWENet) trainable end-to-end and based on an encoder–decoder architecture for modeling the robust feature statistics in individual feature subspaces. We introduce a subspace encoder block at the end of the encoder stage and divide the entire feature space into a set of subspaces; the second-order statistics of all subspaces are concatenated. Then, the wavelet pooling block, suppressing the noise and keeping the structures of learned features well, decomposes the features into low-frequency (storing the basic object structures) and high-frequency components by Haar wavelet layer (HWL), and this block reconstructs the processed components using inverse IHWL during the upsampling stage. Especially, the wavelet pooling block is defined in each subspace for powerful feature learning. Experimental results on a TerraSAR-X image classification dataset suggest that our proposed SWENet yields a performance boost over its competitors.

Latent space models for multiplex networks with shared structure

Summary Latent space models are frequently used for modelling single-layer networks and include many popular special cases, such as the stochastic block model and the random dot product graph. However, they are not well developed for more complex network structures, which are becoming increasingly common in practice. In this article we propose a new latent space model for multiplex networks, i.e., multiple heterogeneous networks observed on a shared node set. Multiplex networks can represent a network sample with shared node labels, a network evolving over time, or a network with multiple types of edges. The key feature of the proposed model is that it learns from data how much of the network structure is shared between layers and pools information across layers as appropriate. We establish identifiability, develop a fitting procedure using convex optimization in combination with a nuclear-norm penalty, and prove a guarantee of recovery for the latent positions provided there is sufficient separation between the shared and the individual latent subspaces. We compare the model with competing methods in the literature on simulated networks and on a multiplex network describing the worldwide trade of agricultural products.

On Methods and Measures for the Inspection of Arbitrarily Oriented Subspace Clusters

When using arbitrarily oriented subspace clustering algorithms one obtains a partitioning of a given data set and for each partition its individual subspace. Since clustering is an unsupervised machine learning task, we may not have “ground truth” labels at our disposal or do not wish to rely on them. What is needed in such cases are internal measure which permits a label-less analysis of the obtained subspace clustering. In this work, we propose methods for revising clusters obtained from arbitrarily oriented correlation clustering algorithms. Initial experiments conducted reveal improvements in the clustering results compared to the original clustering outcome. Our proposed approach is simple and can be applied as a post-processing step on arbitrarily oriented correlation clusterings.

A DBSCAN-based framework to mine travel patterns from origin-destination matrices: Proof-of-concept on proxy static OD from Brisbane

Limited studies exist in the literature on demand related travel patterns, the analysis of which requires a rich database of Origin Destination (OD) matrices with appropriate clustering algorithms. This paper develops a methodological framework to explore typical travel patterns from multi-density high dimensional matrices and estimate typical OD corresponding to those patterns. The contributions of the paper are multi-fold. First, to cluster high-dimensional OD matrices, we deploy geographical window-based structural similarity index (GSSI) as proximity measure in the DBSCAN algorithm that captures both OD structure and network related attributes. Second, to address the issue of multi-density data points, we propose clustering on individual subspaces. Third, we develop a simple two-level approach to identify optimum DBSCAN parameters. Finally, as proof-of-concept, the proposed framework is applied on proxy OD matrices from real Bluetooth data (B-OD) from Brisbane City Council region. The OD matrix clusters, typical travel patterns, and typical B-OD matrices are estimated for this study region. The analysis reveals nine typical travel patterns. The methodology was also found to perform better when GSSI was used instead of Euclidian distance as a proximity measure, and two-level DBSCAN instead of K-medoids, Spectral, and Hierarchical methods. The framework is generic and applicable for OD matrices developed from other data sources and any spatiotemporal context. DBSCAN is chosen for this study because it does not require a pre-determined number of clusters, and it identifies outliers as noise.

Granular Fuzzy Modeling Guided Through the Synergy of Granulating Output Space and Clustering Input Subspaces

As an augmentation of classic fuzzy models, granular fuzzy models (GFMs) have been applied to many fields being in rapport with experimental data, models, and users. However, most of the existing methods used to construct GFMs are based on the principle of optimal allocation of information granularity, which requires that a numeric model be provided in advance. In this paper, a straightforward and convincing modeling method is proposed to directly construct GFM on a basis of experimental data. The method first granulates the output space to form some interval information granules with distinct semantics and then uses them to partition the entire input space into a series of input subspaces. Subsequently, an initial GFM is emerged by using "If-Then" rules to relate with those interval information granules positioned in the output space and structures expressed in prototypes that are produced by clustering individual input subspaces. Further, the initial GFM is also refined by continuously migrating prototypes in individual input subspaces. The experimental studies using the synthetic dataset and several real-world datasets are reported. They offer a useful insight into the feasibility and effectiveness of the proposed modeling method and reveal the impact of parameters on the performance of the ensuing GFMs. An application example is also presented to exhibit the advantages of the resulting GFM.

Second-order encoding networks for semantic segmentation

Recently most of the state-of-the-art semantic segmentation methods have focused on context modeling for more accurate prediction. As real-world images often contain multiple objects and stuff, image features may have complex and multi-modal distributions. However, existing methods do not fully consider sch complex distributions, having limited capability for context modeling. Towards addressing this problem, this paper proposes a second-order encoding network (SoENet) trainable end-to-end for harvesting complex contextual knowledge. At the core of SoENet is an encoding module which can capture second-order statistics in individual feature subspaces. Specifically, we divide the entire feature space into a set of subspaces (clusters) represented by codewords, in each of which a covariance matrix is computed for second-order statistical modeling. The covariance matrices of all subspaces are concatenated to form a 3D tensor, which is then subject to convolutions and nonlinear activations and finally used for scaling of input features. In this way, we can encode the context which involves the complex distribution into learning process in an end-to-end manner. The proposed SoENet is evaluated on four commonly used challenging benchmarks, i.e., PASCAL Context, PASCAL VOC 2012, ADE20K and Cityscapes. The experiments show that our network significantly outperforms its counterparts and is competitive compared to state-of-the-art methods.

Nonlinear Deformation Synthesis via Sparse Principal Geodesic Analysis

AbstractThis paper introduces the construction of a low‐dimensional nonlinear space capturing the variability of a non‐rigid shape from a data set of example poses. The core of the approach is a Sparse Principal Geodesic Analysis (SPGA) on the Riemannian manifold of discrete shells, in which a pose of a non‐rigid shape is a point. The SPGA is invariant to rigid body motions of the poses and supports large deformation. Since the Riemannian metric measures the membrane and bending distortions of the shells, the sparsity term forces the modes to describe largely decoupled and localized deformations. This property facilitates the analysis of articulated shapes. The modes often represent characteristic articulations of the shape and usually come with a decomposing of the spanned subspace into low‐dimensional widely decoupled subspaces. For example, for human models, one expects distinct, localized modes for the bending of elbow or knee whereas some more modes are required to represent shoulder articulation. The decoupling property can be used to construct useful starting points for the computation of the nonlinear deformations via a superposition of shape submanifolds resulting from the decoupling. In a preprocessing stage, samples of the individual subspaces are computed, and, in an online phase, these are interpolated multilinearly. This accelerates the construction of nonlinear deformations and makes the method applicable for interactive applications. The method is compared to alternative approaches and the benefits are demonstrated on different kinds of input data.

A fuzzy‐clustering‐based hierarchical i‐vector/probabilistic linear discriminant analysis system for text‐dependent speaker verification

AbstractIn the i‐vector/probabilistic linear discriminant analysis (PLDA) technique, the PLDA backend classifier is modelled on i‐vectors. PLDA defines an i‐vector subspace that compensates the unwanted variability and helps to discriminate among speaker‐phrase pairs. The channel or session variability manifested in i‐vectors are known to be nonlinear in nature. PLDA training, however, assumes the variability to be linearly separable, thereby causing loss of important discriminating information. Besides, the i‐vector estimation, itself, is known to be poor in case of short utterances. This paper attempts to address these issues using a simple hierarchy‐based system. A modified fuzzy‐clustering technique is employed to divide the feature space into more characteristic feature subspaces using vocal source features. Thereafter, a separate i‐vector/PLDA model is trained for each of the subspaces. The sparser alignment owing to subspace‐specific universal background model and the relatively reduced dimensions of variability in individual subspaces help to train more effective i‐vector/PLDA models. Also, vocal source features are complementary to mel frequency cepstral coefficients, which are transformed into i‐vectors using mixture model technique. As a consequence, vocal source features and i‐vectors tend to have complementary information. Thus using vocal source features for classification in a hierarchy tree may help to differentiate some of the speaker‐phrase classes, which otherwise are not easily discriminable based on i‐vectors. The proposed technique has been validated on Part 1 of RSR2015 database, and it shows a relative equal error rate reduction of up to 37.41% with respect to the baseline i‐vector/PLDA system.

Geometric morphometrics reveals increased symmetric shape variation and asymmetry related to lead exposure in the freshwater green alga Micrasterias compereana

Lead (Pb) contamination of surface waters caused by its gradual release from soils has been reported in acidic wetlands located near industrial regions that experienced significant atmospheric Pb deposition during the 19th and 20th centuries. The green algae belonging to Desmidiales are typically one of the dominant components of the phytobenthos in these habitats, and experimental studies have shown that morphogenesis of desmid cells can be significantly affected by increased Pb concentrations in the environment. This study focused on an analysis of the shape variation in cells of the model desmid species Micrasterias compereana exposed to low chronic levels of Pb contamination. The geometric morphometrics of biradial symmetry and asymmetry of cellular lobes detected a significant increase in the shape variation corresponding to the Pb exposure of populations. However, the sensitivity of this relation was considerably different in the individual subspaces of the symmetric variation and asymmetry. The symmetric variation of the lateral lobes among the cells and the asymmetry between two opposite semicells were the most sensitive components to Pb exposure. Conversely, the left–right asymmetry between two cellular halves across the vertical axis and the transversal asymmetry between the lateral lobes were considerably less sensitive. In addition, the bilateral fluctuating asymmetry in the shapes of two adjacent cellular lobules was also susceptible to increased Pb contamination. Thus, the results showed that quantitative changes in the morphological variation of desmid populations evaluated by the biradial symmetry and asymmetry of cells might be used as sensitive biological indicators for the detection of Pb pollution in dystrophic freshwater habitats, such as peatbogs and acidic fens.