Alternating Direction Minimization Research Articles

Low-Rank Tensor Regularized Views Recovery for Incomplete Multiview Clustering.

In real applications, it is often that the collected multiview data contain missing views. Most existing incomplete multiview clustering (IMVC) methods cannot fully utilize the underlying information of missing data or sufficiently explore the consistent and complementary characteristics. In this article, we propose a novel Low-rAnk Tensor regularized viEws Recovery (LATER) method for IMVC, which jointly reconstructs and utilizes the missing views and learns multilevel graphs for comprehensive similarity discovery in a unified model. The missing views are recovered from a common latent representation, and the recovered views conversely improve the learning of shared patterns. Based on the shared subspace representations and recovered complete multiview data, the multilevel graphs are learned by self-representation to fully exploit the consistent and complementary information among views. Besides, a tensor nuclear norm regularizer is introduced to pursue the global low-rank property and explore the interview correlations. An alternating direction minimization algorithm is presented to optimize the proposed model. Moreover, a new initialization method is proposed to promote the effectiveness of our method for latent representation learning and missing data recovery. Extensive experiments demonstrate that our method outperforms the state-of-the-art approaches.

Generalized latent multi-view clustering with tensorized bipartite graph

Tensor-based multi-view spectral clustering algorithms use tensors to model the structure of multi-dimensional data to take advantage of the complementary information and high-order correlations embedded in the graph, thus achieving impressive clustering performance. However, these algorithms use linear models to obtain consensus, which prevents the learned consensus from adequately representing the nonlinear structure of complex data. In order to address this issue, we propose a method called Generalized Latent Multi-View Clustering with Tensorized Bipartite Graph (GLMC-TBG). Specifically, in this paper we introduce neural networks to learn highly nonlinear mappings that encode nonlinear structures in graphs into latent representations. In addition, multiple views share the same latent consensus through nonlinear interactions. In this way, a more comprehensive common representation from multiple views can be achieved. An Augmented Lagrangian Multiplier with Alternating Direction Minimization (ALM-ADM) framework is designed to optimize the model. Experiments on seven real-world data sets verify that the proposed algorithm is superior to state-of-the-art algorithms.

A combined first and fractional order regularization method for mixed Poisson-white spike noisy image restoration

In this paper we study the problem of restoring images affected by mixed Poisson-Square Cauchy noise. A multi-convex variational model is derived by integrating infimal convolution likelihood into a process of joint maximum a posteriori estimation, where a modified four-directional fractional-order total variation is united with the first order total variation to characterize the image prior. To solve the proposed model numerically, a block coefficient descent based algorithm is derived, in which variable splitting and alternating direction minimization of multipliers are utilized along with anisotropic diffusion and additive operator splitting to gain efficiency and quality. The obtained numerical results are compared with results obtained from two other total variation regularized models with different fidelities. The numerical discuss confirms the flexibility and validity of the proposed model.

A convex selective segmentation model based on a piece-wise constant metric-guided edge detector function

The challenge of segmentation for noisy images, especially those that have light in their backgrounds, still exists in many advanced state-of-the-art segmentation models. Furthermore, it is a tedious activity and significantly difficult to segment such images. In this article, we offer and propose a novel variational model for the concurrent restoration and segmentation of noisy images that have intensity inhomogeneity and high contrast background illumination and light. The suggested concept combines the multi-phase segmentation technology with the statistical approach in terms of local region knowledge and details of circular regions that are, in fact, centered at every pixel to enable in-homogeneous image restoration. Nevertheless, the suggested approach is expressed as a fuzzy set and is resolved using the multiplier alternating direction minimization approach. Through several tests and numerical simulations with plausible assumptions, we have evaluated the correctness, accuracy, and resilience of the suggested model over various kinds of real and synthesized images in the existence of intensity inhomogeneity and light in the background. Additionally, the findings are contrasted with those from cutting-edge two-phase and multi-phase methods, proving the superiority of our proposed approach for images with noise, background light, and inhomogeneity. We observed that the three models considered in the comparison to our proposed model fail to properly segment the object of interest. Furthermore, the Jaccard similarity metric shows the capability of our model to handle accurate segmentation of complex shapes with intensity.

Graph-Guided Unsupervised Multiview Representation Learning

Without the valuable label information to guide the learning process, it is demanding to fully excavate and integrate the underlying information from different views to learn the unified multi-view representation. This paper focuses on this challenge and presents a novel method, termed Graph-guided Unsupervised Multi-view Representation Learning (GUMRL), taking full advantage of multi-view graph information during the learning process. To be specific, GUMRL jointly conducts the view-specific feature representation learning, which is under the guidance of graph information, and the unified feature representation learning, which fuses the underlying graph information of different views to learn the desired unified multi-view feature representation. Regarding downstream tasks, such as clustering and classification, the classic single-view algorithms can be directly performed on the learned unified multi-view representation. The designed objective function is effectively optimized based on an alternating direction minimization method, and experiments conducted on six real-world multi-view datasets show the effectiveness and competitiveness of our GUMRL, compared to several state-of-the-art methods.

Multi-view clustering with orthogonal mapping and binary graph

The multi-view clustering problem has attracted considerable attention over recent years for the remarkable clustering performance due to exploiting complementary information from multiple views. Most existing related research work processes data in the decimal real value space that is not the most compatible space for computers. Binary code learning, also known as hashing technology, is well-known for fast Hamming distance computation, less storage requirement and accurate calculation results. The Hamming space is most enjoyed by computers because of binary/hash codes. Several studies combine multi-view clustering with binary code learning for improving clustering performance. However, there is much redundant information contained in the learned binary codes, which negatively affects the clustering performance, but these studies ignore eliminating redundant information for learning compact codes. In addition, they don’t give a unified (one-step) clustering framework with binary graph structure, which doesn’t lead to the optimal clustering result due to the information loss during the two-step process. In this paper, to cope with the two issues, we propose an orthogonal mapping binary graph method (OMBG) for the multi-view clustering problem, which makes the mapping matrix of every view orthogonalize for eliminating redundant information and embeds a binary graph structure into the unified binary multi-view clustering framework for extracting local geometric structure information of binary codes and achieving the optimal clustering result. Furthermore, we design an effective optimization algorithm based on alternating direction minimization to solve the model of OMBG. Extensive experiments performed on four frequently-used benchmark multi-view datasets illustrate the superiority of OMBG which is compared with some state-of-the-art clustering baselines.

Multi-scale-average-filter-assisted level set segmentation model with local region restoration achievements

Segmentation of noisy images having light in the background it is a challenging task for the existing segmentation approaches and methods. In this paper, we suggest a novel variational method for joint restoration and segmentation of noisy images which are having intensity and inhomogeneity in the existence of high contrast light in the background. The proposed model combines statistical local region information of circular regions centered at each pixel with a multi-phase segmentation technique enabling inhomogeneous image restoration. The proposed model is written in the fuzzy set framework and resolved through alternating direction minimization approach of multipliers. Through experiments, we have tested the performance of the suggested approach on diverse types of synthetic and real images in the existence of intensity and in-homogeneity; and evaluate the precision, as well as, the robustness of the suggested model. Furthermore, the outcomes are, then, compared with other state-of-the-art models including two-phase and multi-phase approaches and show that our method has superiority for images in the existence of noise and inhomogeneity. Our empirical evaluation and experiments, using real images, evaluate and assess the efficiency of the suggested model against several other closest rivals. We observed that the suggested model can precisely segment all the images having brightness, diffuse edges, high contrast light in the background, and inhomogeneity.

One-Step Multiview Subspace Segmentation via Joint Skinny Tensor Learning and Latent Clustering

Multiview subspace clustering (MSC) has attracted growing attention due to the extensive value in various applications, such as natural language processing, face recognition, and time-series analysis. In this article, we are devoted to address two crucial issues in MSC: 1) high computational cost and 2) cumbersome multistage clustering. Existing MSC approaches, including tensor singular value decomposition (t-SVD)-MSC that has achieved promising performance, generally utilize the dataset itself as the dictionary and regard representation learning and clustering process as two separate parts, thus leading to the high computational overhead and unsatisfactory clustering performance. To remedy these two issues, we propose a novel MSC model called joint skinny tensor learning and latent clustering (JSTC), which can learn high-order skinny tensor representations and corresponding latent clustering assignments simultaneously. Through such a joint optimization strategy, the multiview complementary information and latent clustering structure can be exploited thoroughly to improve the clustering performance. An alternating direction minimization algorithm, which owns low computational complexity and can be run in parallel when solving several key subproblems, is carefully designed to optimize the JSTC model. Such a nice property makes our JSTC an appealing solution for large-scale MSC problems. We conduct extensive experiments on ten popular datasets and compare our JSTC with 12 competitors. Five commonly used metrics, including four external measures (NMI, ACC, F-score, and RI) and one internal metric (SI), are adopted to evaluate the clustering quality. The experimental results with the Wilcoxon statistical test demonstrate the superiority of the proposed method in both clustering performance and operational efficiency.

A unified total variation method for underwater image enhancement

Underwater images usually suffer from color casts and low contrast due to the absorption and scattering of light by the water medium. The degradation is caused not only by the light attenuation on the scene-sensor path but also by the light attenuation on the water surface-scene path. To eliminate the dual-path light attenuation, we propose a novel unified total variation method based on an extended underwater imaging model. Unlike previous variation-based methods that only consider light propagation along the scene-sensor path, we additionally include light propagation along the water surface-scene path in the underwater imaging model. In the proposed variational framework, we transform underwater image enhancement into two subproblems and construct different prior knowledge-guided optimization functions for them. The two subproblems aim to remove the light attenuation along the scene-sensor and surface-scene paths. Moreover, we present an alternating direction minimization algorithm based on an augmented Lagrange multiplier to address the optimization problems. The subjective and objective experimental results on underwater images with different attenuation characteristics demonstrate that the proposed method achieves good performance in underwater image enhancement.

Tensor-based multi-view clustering with consistency exploration and diversity regularization

How to make good use of information from all views is the core problem in multi-view clustering (MVC), and widely recognized experience of attaining this goal is to leverage consistent and complementary principles. However, current methods do not take full advantage of these two principles. On the one hand, some existing methods ignore the connection between multiple views and the influence of manifold information on consensus representation. On the other hand, some others simply sum all the view-specific representations together as the final representation, which result in insufficient mining of complementary information. In this manuscript, we propose a novel tensor-based multi-view clustering approach that significantly improves consistency exploration and diversity capturing. In particular, we utilize the low-rank tensor learning with tensor-based singular value decomposition (t-SVD), aiming at capturing the correlations among different views. Moreover, we further split the low-rank tensor into a consistent matrix and a set of view-specific matrices, where the former is responsible for exploring the common manifold information via graph regularization, and the latter is used to mine complementary information through learning the view-specific representations as various as possible. We integrate low-rank tensor learning, consistency exploration and diversity regularization into a whole framework and use an alternating direction minimization strategy to optimize it. Experiments conducted on eight benchmark datasets show that our approach gains superior performance over several state-of-the-arts.

Counting Objects by Diffused Index: Geometry-free and training-free approach

Counting objects is a fundamental but challenging problem. In this paper, we propose diffusion-based, geometry-free, and learning-free methodologies to count the number of objects in images. The main idea is to represent each object by a unique index value regardless of its intensity or size, and to simply count the number of index values. First, we place different vectors, refer to as seed vectors, uniformly throughout the mask image. The mask image has boundary information of the objects to be counted. Secondly, the seeds are diffused using an edge-weighted harmonic variational optimization model within each object. We propose an efficient algorithm based on an operator splitting approach and alternating direction minimization method, and theoretical analysis of this algorithm is given. An optimal solution of the model is obtained when the distributed seeds are completely diffused such that there is a unique intensity within each object, which we refer to as an index. For computational efficiency, we stop the diffusion process before a full convergence, and propose to cluster these diffused index values. We refer to this approach as Counting Objects by Diffused Index (CODI). We explore scalar and multi-dimensional seed vectors. For Scalar seeds, we use Gaussian fitting in histogram to count, while for vector seeds, we exploit a high-dimensional clustering method for the final step of counting via clustering. The proposed method is flexible even if the boundary of the object is not clear nor fully enclosed. We present counting results in various applications such as biological cells, agriculture, concert crowd, and transportation. Some comparisons with existing methods are presented.

Field of experts regularized nonlocal low rank matrix approximation for image denoising

The restoration of image degraded by noise is an essential preprocessing step for various imaging technologies. Nonlocal low rank matrix approximation has been successfully applied to image denoising due to the capability of recovering the underlying low rank structures. Unfortunately, existing rank minimization models ignore the correlation among image patches and their performance is degraded when encountering the heavy noise. To address this, we propose a field of experts regularized nonlocal low rank matrix approximation (RFoE) denoising model, which integrates a global field of experts (FoE) regularization, a fidelity term, and a nonlocal low rank constraint into a unified framework. The weighted nuclear norm is adopted as the low rank constraint while the FoE prior is utilized to capture the global structure information. An alternating direction minimization algorithm based on half quadratic splitting can effectively solve this model. Extensive experimental results demonstrate that the proposed RFoE model has a superior performance.

Representative Task Self-Selection for Flexible Clustered Lifelong Learning.

Consider the lifelong machine learning paradigm whose objective is to learn a sequence of tasks depending on previous experiences, e.g., knowledge library or deep network weights. However, the knowledge libraries or deep networks for most recent lifelong learning models are of prescribed size and can degenerate the performance for both learned tasks and coming ones when facing with a new task environment (cluster). To address this challenge, we propose a novel incremental clustered lifelong learning framework with two knowledge libraries: feature learning library and model knowledge library, called Flexible Clustered Lifelong Learning (FCL3). Specifically, the feature learning library modeled by an autoencoder architecture maintains a set of representation common across all the observed tasks, and the model knowledge library can be self-selected by identifying and adding new representative models (clusters). When a new task arrives, our FCL3 model firstly transfers knowledge from these libraries to encode the new task, i.e., effectively and selectively soft-assigning this new task to multiple representative models over feature learning library. Then: 1) the new task with a higher outlier probability will be judged as a new representative, and used to redefine both feature learning library and representative models over time; or 2) the new task with lower outlier probability will only refine the feature learning library. For model optimization, we cast this lifelong learning problem as an alternating direction minimization problem as a new task comes. Finally, we evaluate the proposed framework by analyzing several multitask data sets, and the experimental results demonstrate that our FCL3 model can achieve better performance than most lifelong learning frameworks, even batch clustered multitask learning models.

Constrained Tensor Representation Learning for Multi-View Semi-Supervised Subspace Clustering

Multi-view subspace clustering is an effective method to partition data into their corresponding categories. Nevertheless, existing multi-view subspace clustering approaches generally operate in a purely unsupervised manner, while ignoring the valuable weakly supervised information that can be readily obtained in many practical applications. In this paper, we consider the weakly supervised form of sample pair constraints, and devote to promoting the performance of multi-view subspace clustering with the aid of such prior knowledge. To achieve this goal, inspired by the intrinsic block diagonal structure of ideal low-rank representation (LRR), we propose a novel regularization to integrate must-link, cannot-link and normalization constraints into a unified formulation. The proposed regularization can be regarded as a general description for sample pairwise constraints, and thus provides a flexible framework for multi-view semi-supervised subspace clustering task. Furthermore, we devise a contrained tensor representation learning (CTRL) model that takes advantage of our proposed regularization to facilitate the learning of the desired representation tensor. An efficient optimization algorithm based on alternating direction minimization strategy is carefully designed to solve the proposed CTRL model. Extensive experiments on eight challenging real-world datasets are conducted, and the results validate the effectiveness of our designed pairwise constraints regularization, as well as the superiority of the proposed CTRL model.

Non-convex fractional-order derivative for single image blind restoration

This paper considers a variational model for single image blind restoration. By exploiting the fractional-order total variation (FTV) and Lp quasinorm relaxation, a non-convex fractional-order variational model is proposed to restore the image blurred by an unknown blur kernel. A quaternion FTV model is first put forward to exploring more directional information of an image. The new model utilizes non-convex and non-smooth quaternion FTV with Lp quasinorm to constrain the image and L1-norm to constrain the blur kernel, which are unified to a unified regularization framework. Further, an efficient algorithm is proposed to solve the non-convex problem via using the alternating direction minimization. The extensive experiments demonstrate the efficiency and viability of the proposed method and reveal superior performances in comparison with several existing methods.

Robust Online Prediction of Spectrum Map With Incomplete and Corrupted Observations

Spectrum map is an essential tool for a range of emerging applications of 5G and 6G networks. Despite the great efforts that have been put on the construction of spectrum maps, access to accurate and valid spectrum data in dynamically changing environments emphasizes the need for more advanced solutions tailored to such rapidly varying scenarios. To this end, the idea of spectrum map prediction is introduced. In this paper, we address the problem of spectrum map prediction from historical spectrum observations in the dynamically changing environments. The problem is particularly challenging when the available historical spectrum observations are incomplete and corrupted by anomalies. We propose three techniques to solve the problem. First, we combine the spectrum map with prediction functionalities so as to offer a huge potential for efficient resource management and flexible sharing of resources in dynamically changing environments. Second, by fully exploiting the hidden spatial-temporal-spectral structures of the spectrum data and the sparsity of anomalies and missing data, we model the spectrum map as a 3rd-order spectrum tensor and formulate the spectrum map prediction problem as a low-rank tensor completion problem. Third, we design a robust online spectrum map prediction (ROSMP) algorithm based on the alternating direction minimization method, which derives the tensor decomposition factors for a new timeslot based on the update of existing ones rather than re-computing from the scratch. By gradually learning the hidden spatial-temporal-spectral structures of the spectrum data, ROSMP is able to predict and obtain the complete spectrum map with high accuracy. Finally, extensive numerical evaluations using a real spectrum measurement dataset confirm the efficacy and efficiency of ROSMP and show the superiority of ROSMP over the baselines.

An Alternating Direction Minimization based denoising method for extracted ion chromatogram

Accurate extracted ion chromatograms (XIC or EIC) is of great importance for Liquid chromatography - mass spectrometry (LC-MS) based quantitative proteomics. However, current preprocessing methods for XIC mainly focus on removing the peaks of low quality. Such operations are helpful for quantitation, but also contain two potential disadvantages: one is that some valuable information may be lost after the peak removal, and the other is that the retained peak lists are still contain noise. Both of the potential disadvantages may bias the final quantitative results. To solve these problems, we proposed an Alternating Direction Minimization (ADM) based denoising framework for XIC. This framework splits the observed XIC signal into baseline (background), noise and true signal, and the true signal is extracted for further analysis. The advantage of this framework is that the inner relationships over each XIC are considered, for which means that the XIC data is handled in a global way. Also, this framework is not sensitive to the noise type, so that it can be applied to wider applications. We adopted the framework for some sample data for quantitation. The quantitative results are employed to show the performance of XIC denoising. Experimental results confirm that the proposed method provides better and more reliable quantitations.

Latent Low-Rank Sparse Multi-view Subspace Clustering

To solve the problem of multi-view clustering,a latent low-rank sparse multi-view subspace clustering(LLSMSC)algorithm is proposed.A latent space shared by all views is constructed to explore the complementary information of multi-view data.The global and local structure of multi-view data can be captured to attain promising clustering results by imposing low-rank constraint and sparse constraint on the implicit latent subspace representation simultaneously.An algorithm based on augmented Lagrangian multiplier with alternating direction minimization strategy is employed to solve the optimization problem.Experiments on six benchmark datasets verify the effectiveness and superiority of LLSMSC.

Tensorized Multi-view Subspace Representation Learning

Self-representation based subspace learning has shown its effectiveness in many applications. In this paper, we promote the traditional subspace representation learning by simultaneously taking advantages of multiple views and prior constraint. Accordingly, we establish a novel algorithm termed as Tensorized Multi-view Subspace Representation Learning. To exploit different views, the subspace representation matrices of different views are regarded as a low-rank tensor, which effectively models the high-order correlations of multi-view data. To incorporate prior information, a constraint matrix is devised to guide the subspace representation learning within a unified framework. The subspace representation tensor equipped with a low-rank constraint models elegantly the complementary information among different views, reduces redundancy of subspace representations, and then improves the accuracy of subsequent tasks. We formulate the model with a tensor nuclear norm minimization problem constrained with $$\ell _{2,1}$$ -norm and linear equalities. The minimization problem is efficiently solved by using an Augmented Lagrangian Alternating Direction Minimization method. Extensive experimental results on diverse multi-view datasets demonstrate the effectiveness of our algorithm.

Adaptive latent similarity learning for multi-view clustering

Most existing clustering methods employ the original multi-view data as input to learn the similarity matrix which characterizes the underlying cluster structure shared by multiple views. This reduces the flexibility of multi-view clustering methods due to the fact that multi-view data usually contains noise or the variation between multi-view data points, which should belong to the same cluster, is larger than the variation between data points belonging to different clusters. To address these problems, we propose a novel multi-view clustering model, namely adaptive latent similarity learning (ALSL) for multi-view clustering. ALSL employs the adaptively learned graph, which characterizes the relationship between clusters, as the new input to learn the latent data representation and integrates the latent similarity representation learning, manifold learning and spectral clustering into a unified framework. With the complementarity of multiple views, the latent similarity representation characterizes the underlying cluster structure shared by multiple views. Our model is intuitive and can be optimized efficiently by using the Augmented Lagrangian Multiplier with Alternating Direction Minimization (ALM-ADM) algorithm. Extensive experiments on benchmark datasets have demonstrated the superiority of the proposed method.