Manifold Constraints Research Articles

Manifold Constrained Low-Rank and Joint Sparse Learning for Dynamic Cardiac MRI

Reconstruction from highly accelerated dynamic magnetic resonance imaging (MRI) is of great significance for medical diagnosis. The application of low-rank and sparse matrix decomposition to MRI can improve imaging speed and efficiency. However, the consistence of the learned low-rank and sparse structures for similar input samples is not well addressed in literature. In this paper, we propose a manifold constrained low-rank and joint sparse learning model that embeds the manifold priors into low-rank and joint sparse decomposition. It is noted that the joint sparsity is investigated to exploit the shared information. Further, the manifold constraints for low-rank and joint sparse parts are forced the optimization process to satisfy the structure preservation requirement. To solve the above manifold learning problem, a manifold constrained alternating direction method of multipliers ( McADMM ) approach is designed. It is proved theoretically that the sequence generated by McADMM converges to a stationary point. Numerical comparisons on simulation data and real-world dynamic cardiac MRI data are presented to demonstrate its efficiency.

Sparse graphs using global and local smoothness constraints

Graph-based learning methods are very useful for many machine learning approaches and classification tasks. Constructing an informative graph is one of the most important steps since a graph can significantly affect the final performance of the learning algorithms. Sparse representation is a useful tool in machine learning and pattern recognition area. Recently, it was shown that sparse graphs (sparse representation based graphs) provide a powerful approach to graph-based semi-supervised classification. In this paper, we introduce a new graph construction method that simultaneously provides a sparse graph and integrates manifold constraints on the sparse coefficients without any prior knowledge on the graph or on its similarity matrix. Furthermore, we propose an efficient solution to the optimization problem. The proposed method imposes that the sparse coding vectors of similar samples should be also similar. Different from existing graph construction methods that are based on the use of explicit constraints or a predefined graph matrix, the proposed smoothness constraints on the graph weights implicitly adapt data to the global structure of the estimated graph. A series of experiments conducted on several public image databases shows that the proposed method can outperform many state-of-the-art methods when applied to the problem of graph-based label propagation.

Exploiting Embedding Manifold of Autoencoders for Hyperspectral Anomaly Detection

Hyperspectral anomaly detection is an important task in the remote sensing domain. Recently, researchers have shown great interest in deep learning-based methods because they can learn hierarchical, abstract, and high-level representations. However, the latent features learned from the autoencoder (AE) are not always able to reflect the intrinsic structure of hyperspectral data because the locality property is not considered during the learning process. In order to address this problem, a novel manifold constrained AE network (MC-AEN)-based hyperspectral anomaly detection method is proposed in this article. First, the manifold learning method is employed to learn the embedding manifold. Then, the latent representations are learned by an AE network with the learned embedding manifold constraints to preserve the intrinsic structure of hyperspectral data. Finally, the reconstruction errors are calculated to detect anomalies. The global reconstruction error from MC-AEN and the local reconstruction error from the learned latent representations are combined to fully utilize the learned knowledge for better detection performance. We test our proposed algorithm on three different real data sets. Experimental results on these three data sets show the superiority of our proposed method.

Exploring implicit spaces for constrained sampling-based planning

We present a review and reformulation of manifold constrained sampling-based motion planning within a unifying framework, IMACS (implicit manifold configuration space). IMACS enables a broad class of motion planners to plan in the presence of manifold constraints, decoupling the choice of motion planning algorithm and method for constraint adherence into orthogonal choices. We show that implicit configuration spaces defined by constraints can be presented to sampling-based planners by addressing two key fundamental primitives, sampling and local planning, and that IMACS preserves theoretical properties of probabilistic completeness and asymptotic optimality through these primitives. Within IMACS, we implement projection- and continuation-based methods for constraint adherence, and demonstrate the framework on a range of planners with both methods in simulated and realistic scenarios. Our results show that the choice of method for constraint adherence depends on many factors and that novel combinations of planners and methods of constraint adherence can be more effective than previous approaches. Our implementation of IMACS is open source within the Open Motion Planning Library and is easily extended for novel planners and constraint spaces.

Robust High-Order Manifold Constrained Sparse Principal Component Analysis for Image Representation

In order to efficiently utilize the information in the data and eliminate the negative effects of outliers in the principal component analysis (PCA) method, in this paper, we propose a novel robust sparse PCA method based on maximum correntropy criterion (MCC) with high-order manifold constraints called the RHSPCA. Compared with the traditional PCA methods, the proposed RHSPCA has the following benefits: 1) the MCC regression term is more robust to outliers than the MSE-based regression term; 2) thanks to the high-order manifold constraints, the low-dimensional representations can preserve the local relations of the data and greatly improve the clustering and classification performance for image processing tasks; and 3) in order to further counteract the adverse effects of outliers, the MCC-based samples’ mean is proposed to better centralize the data. We also propose a new solver based on the half-quadratic technique and accelerated block coordinate update strategy to solve the RHSPCA model. Extensive experimental results show that the proposed method can outperform the state-of-the-art robust PCA methods on a variety of image processing tasks, including reconstruction, clustering, and classification, on outliers contaminated datasets.

Sparse graphs with smoothness constraints: Application to dimensionality reduction and semi-supervised classification

Sparse representation is a useful tool in machine learning and pattern recognition area. Sparse graphs (graphs constructed using sparse representation of data) proved to be very informative graphs for many learning tasks such as label propagation, embedding, and clustering. It has been shown that constructing an informative graph is one of the most important steps since it significantly affects the final performance of the post graph-based learning algorithm. In this paper, we introduce a new sparse graph construction method that integrates manifold constraints on the unknown sparse codes as a graph regularizer. These constraints seem to be a natural regularizer that was discarded in existing state-of-the art graph construction methods. This regularizer imposes constraints on the graph coefficients in the same way a locality preserving constraint imposes on data projection in non-linear manifold learning. The proposed method is termed Sparse Graph with Laplacian Smoothness (SGLS). We also propose a kernelized version of the SGLS method. A series of experimental results on several public image datasets show that the proposed methods can out-perform many state-of-the-art methods for the tasks of label propagation, nonlinear and linear embedding.

Electroencephalography Classification in Brain-Computer Interface with Manifold Constraints Transfer.

Insufficient training data is a serious problem in all domains related to bioinformatics. Transfer learning is a promising tool to solve this problem, which relaxes the hypothesis that training data must be independent and identically distributed with the test data. We construct a sophisticated electroencephalography (EEG) signal representation and obtain an efficient EEG feature extractor through manifold constraints-based joint adversarial training with training data from other domains. EEG signal is more easily distinguished in the feature space mapped by the feature extractor. Negative transfer is one of the most challenging problems in transfer learning. In our approach, we apply manifold constraints to overcome this problem, which can avoid the geometric manifolds in the target domain being destroyed. The experiments demonstrate that our approach has many advantages when applied to EEG classification tasks.

Face recognition based on manifold constrained joint sparse sensing with K-SVD

Face recognition based on Sparse representation idea has recently become an important research topic in computer vision community. However, the dictionary learning process in most of the existing approaches suffers from the perturbations brought by the variations of the input samples, since the consistence of the learned dictionaries from similar input samples based on K-SVD are not well addressed in the existing literature. In this paper, we will propose a novel technique for dictionary learning based on K-SVD to address the consistence issue. In particular, the proposed method embeds the manifold constraints into a standard dictionary learning framework based on k-SVD and force the optimization process to satisfy the structure preservation requirement. Therefore, this new approach can consistently integrate the manifold constraints during the optimization process, and it can contribute a better solution which is robust to the variance of the input samples. Extensive experiments on several popular face databases show a consistent performance improvement in comparison to some related state-of-the-art algorithms.

Manifold constraint transfer for visual structure-driven optimization

In this paper, we leverage the manifold structure of visual data in order to improve performance in general optimization problems subject to linear constraints. As the main theoretical result, we show that manifold constraints can be transferred from the data to the optimized variables if these are linearly correlated. We also show that the resulting optimization problem can be solved with an efficient alternating direction method of multipliers that can consistently integrate the manifold constraints during the optimization process. This leads to a simplification of the approach, which instead of directly optimizing on the manifold, we can iteratively recast the problem as the projection over the manifold via an embedding method. The proposed method is extremely versatile since it can be applied to different problems including Kernel Ridge Regression (KRR) and sparse coding which have numerous applications in machine learning and computer vision. In particular, we apply the methods to different problems such as tracking, object recognition and categorization showing a consistent increase of performance with respect to the state of the art.

ICSSP 2015–Special Issue Introduction

Modern systems consist of a complex mix of products and services—legacy, bespoke, COTS,open source, licensed, and user-developed—that must be designed to meet rapidly changingbusiness needs [9]. Business processes must be agile to capture and maintain customers withrapidly evolving needs, tastes, and desires. Systems technology and development methods must beresponsive to business needs to continuously support value creation. However, legacy and regulatoryconstraints, emerging requirements, enhanced interoperability, and the ascendancy of software havechallenged system development processes, and has made many companies deploying so-calledhybrid development approaches address these manifold constraints [14]. Just as systems are morenecessary for business success, the ability to align systems and business goals and processeshas become more difficult. In fact, in the field of software process development, deployment,and continuous improvement, we find a diversity of (competing) approaches, and we face afield that is still on the quest for the “optimal” approach to organize and improve software andsystem development [8]. In the past, the processes and underlying principles of business, systemengineering and software engineering have independently evolved and are represented by languagesand concepts that are tantalizingly similar and yet seem to be in conflict. This has led to confusionand often friction between and among these critical conceptual constituents [5].The volume at hand presents the special issue of theInternational Conference on Software andSystems Process(ICSSP) 2015, which was held in Tallinn, Estonia from August 24-26, 2015.ICSSP as the premier venue for research related to software and systems development processeshas become an important mediator for industry and academia. To answer questions like the onementioned before, ICSSP brings together scientists and practitioners to discuss problems andsolutions, and to present concrete experiences. Therefore, the six high-quality papers chosen forthis special issue address both a solid scientific background and an industry focus. It is throughpublishing such papers that we can bridge the industry-research gap, ensuring that our research willhave an impact, not only through our teaching and graduate programs but also by having a directeffect on the improvement of processes within industry.

Concise Total Synthesis of (+)-Asperazine, (+)-Pestalazine A, and (+)-iso-Pestalazine A. Structure Revision of (+)-Pestalazine A.

The concise, enantioselective total syntheses of (+)-asperazine (1), (+)-iso-pestalazine A (2), and (+)-pestalazine A (3) have been achieved by the development of a late-stage C3-C8' Friedel-Crafts union of polycyclic diketopiperazines. Our modular strategy enables the union of complex polycyclic diketopiperazines in virtually their final forms, thus providing rapid and highly convergent assembly at the challenging quaternary stereocenter of these dimeric alkaloids. The significance of this carbon-carbon bond formation can be gauged by the manifold constraints that were efficiently overcome, namely the substantial steric crowding at both reactive sites, the nucleophilic addition of C8' over N1' to the C3 carbocation, and the multitude of reactivity posed by the use of complex diketopiperazine fragments in the coupling event. The success of the indoline π-nucleophile that evolved through our studies is notable given the paucity of competing reaction pathways observed in the presence of the highly reactive C3 carbocation generated. This first total synthesis of (+)-pestalazine A also allowed us to revise the molecular structure for this natural alkaloid.

South Africa’s aspirational Constitution and our problems of collective action

This article advances the somewhat novel contention that our Constitution (a) is aspirational in nature; (b) requires three critical preconditions before it delivers on its numerous promises and (c) is subject to manifold constraints that can only ever be partially overcome. It then marries three interlocking theses. The first thesis identifies the manifold constraints on individual, social and political change. These constraints can be overcome. However, change does not occur primarily through deliberation. It’s largely a function of two other processes: friction and experimentation. The second thesis – ‘the basic law qua scaffolding’ – emphasises the limitations of constitutions. The basic law qua scaffolding provides an extremely capacious normative framework within which: (i) political institutions produce public goods; (ii) associations and subpublics provide the meaningful settings for most actions; and (iii) individuals can pursue lives worth valuing. Scaffolding is thus best understood as a process of legitimation in which interpreters of the Constitution confirm (or deny) the validity of decisions taken by other actors within the realm. The third thesis contends that problems of collective action constitute the primary impediment to the production of public goods. The article then entertains proposals about how problems of collective action – politics properly understood – can be overcome.

Image Retrieval Based on Multiview Constrained Nonnegative Matrix Factorization and Gaussian Mixture Model Spectral Clustering Method

Content-based image retrieval has recently become an important research topic and has been widely used for managing images from repertories. In this article, we address an efficient technique, called MNGS, which integrates multiview constrained nonnegative matrix factorization (NMF) and Gaussian mixture model- (GMM-) based spectral clustering for image retrieval. In the proposed methodology, the multiview NMF scheme provides competitive sparse representations of underlying images through decomposition of a similarity-preserving matrix that is formed by fusing multiple features from different visual aspects. In particular, the proposed method merges manifold constraints into the standard NMF objective function to impose an orthogonality constraint on the basis matrix and satisfy the structure preservation requirement of the coefficient matrix. To manipulate the clustering method on sparse representations, this paper has developed a GMM-based spectral clustering method in which the Gaussian components are regrouped in spectral space, which significantly improves the retrieval effectiveness. In this way, image retrieval of the whole database translates to a nearest-neighbour search in the cluster containing the query image. Simultaneously, this study investigates the proof of convergence of the objective function and the analysis of the computational complexity. Experimental results on three standard image datasets reveal the advantages that can be achieved with the proposed retrieval scheme.

Tag Refinement for User-Contributed Images via Graph Learning and Nonnegative Tensor Factorization

Social image tagging systems mostly suffer from poor performance for image retrieval due to the noisy and incomplete correspondences between user-contributed images and their associated tags. In this letter, we aim to refine tag allocations in the social tagging data provided by these systems. In particular, we propose to harness the tagged and untagged data with a two-stage strategy according to different types of data relations, i.e. item similarity defined by prior knowledge and item co-occurrence learned from data statistics. To solve the sparsity problem, we first introduce a new graph learning (GL) method for enriching the tagging data according to item similarities. Then, we develop a method of nonnegative tensor factorization (NTF) for learning more coherent ternary relations among users, images and tags coupled by the manifold constraints learned from item co-occurrences. Experimental results with the tagging data from the NUS-WIDE dataset have been reported to validate the effectiveness of the proposed method.

MR image super-resolution via manifold regularized sparse learning

Single image super-resolution (SR) has been shown useful in Magnetic Resonance (MR) image based diagnosis, where the image resolution is still limited. The basic goal of single image SR is to produce a high-resolution (HR) image from corresponding low-resolution (LR) image. However, most existing SR algorithms often fail to: (1) reflect the intrinsic structure between MR images and (2) exploit the intra-patient information of MR images. In fact, MR images are more likely to vary along a low dimensional submanifold, which can be embedded in the high dimensional space. It has also been shown that the structure information of MR images and the priors of the MR images of different modality are important for improving the image resolution. To take full advantage of manifold structure information and intra-patient prior of MR images, a novel single image super-resolution algorithm for MR images is proposed in this paper. Compared with the existing works, the proposed algorithm has the following merits: (1) the proposed sparse coding based algorithm integrates manifold constraints to handle the inverse problem in MR image SR; (2) the manifold structure of the intra-patient MR image is considered for image SR; and (3) the topological structure of the intra-patient MR image can be preserved to improve the reconstructed result. Experiments show that the proposed algorithm is more effective than the state-of-the-art algorithms.

Beyond the Eagle's Shadow: New Histories of Latin America's Cold War

Beyond the Eagle's Shadow: New Histories of Latin America's Cold War

Semi-supervised domain adaptation on manifolds.

In real-life problems, the following semi-supervised domain adaptation scenario is often encountered: we have full access to some source data, which is usually very large; the target data distribution is under certain unknown transformation of the source data distribution; meanwhile, only a small fraction of the target instances come with labels. The goal is to learn a prediction model by incorporating information from the source domain that is able to generalize well on the target test instances. We consider an explicit form of transformation functions and especially linear transformations that maps examples from the source to the target domain, and we argue that by proper preprocessing of the data from both source and target domains, the feasible transformation functions can be characterized by a set of rotation matrices. This naturally leads to an optimization formulation under the special orthogonal group constraints. We present an iterative coordinate descent solver that is able to jointly learn the transformation as well as the model parameters, while the geodesic update ensures the manifold constraints are always satisfied. Our framework is sufficiently general to work with a variety of loss functions and prediction problems. Empirical evaluations on synthetic and real-world experiments demonstrate the competitive performance of our method with respect to the state-of-the-art.

Circuit-level optimisation of a:Si TFT-based AMOLED pixel circuits for maximum hold current

Design of AMOLED pixel circuits has manifold constraints and trade-offs which provides incentive for circuit designers to seek optimal solutions for different objectives. In this article, we present a discussion on the viability of an optimal solution to achieve the maximum hold current. A compact formula for component sizing in a conventional 2T1C pixel is, therefore, derived. Compared to SPICE simulation results, for several pixel sizes, our predicted optimum sizing yields maximum currents with errors less than 0.4%.

On the use of slow manifolds in molecular and geophysical fluid dynamics

Constraints typically arise from the elimination of high frequency oscillations in mechanical systems. Examples are provided by bond constraints in molecular simulations and incompressibility constraints in fluid dynamics. A key issue is the accuracy of constrained dynamics with regard to the full dynamics. In this review we focus on the smooth solution components and discuss the concept of slow manifold and soft constraints in molecular and geophysical fluid dynamics. While the formal mathematical derivation of constraints is the same for both molecular and fluid dynamics, the predominant numerical techniques for dealing with constraints are different in the two fields. Semi-implicit time- stepping methods are often used in geophysical fluid dynamics while explicitly enforced constraints are more common in molecular dynamics.

MVDR Beamforming for Through-the-Wall Radar Imaging

We present high-definition imaging for targets behind walls and enclosed structures based on constrained minimization RF multisensor processing. Minimum variance distortionless response (MVDR) beamforming is used on both sensor-frequency raw data radar returns and spatial spectrum data, which is obtained by the Fourier transform of the delay and sum beamformer image. We compare both methods for near-field and far-field scenes. The paper considers both cases of known and unknown wall parameters and uses manifold constraints to allow target localization in high-definition imaging in the presence of wall errors. Also, through analyses and simulations, we show how to effectively use the spatial spectrum to improve covariance matrix estimation and subsequently enhance image quality in the sense of lower sidelobes.