Alternating Projection Algorithm Research Articles

Low Rank Pure Quaternion Approximation for Pure Quaternion Matrices

Quaternion matrices are employed successfully in many color image processing applications. In particular, a pure quaternion matrix can be used to represent red, green, and blue channels of color images. A low-rank approximation for a pure quaternion matrix can be obtained by using the quaternion singular value decomposition. However, this approximation is not optimal in the sense that the resulting low-rank approximation matrix may not be pure quaternion, i.e., the low-rank matrix contains a real component which is not useful for the representation of a color image. The main contribution of this paper is to find an optimal rank-$r$ pure quaternion matrix approximation for a pure quaternion matrix (a color image). Our idea is to use a projection on a low-rank quaternion matrix manifold and a projection on a quaternion matrix with zero real component, and develop an alternating projections algorithm to find such optimal low-rank pure quaternion matrix approximation. The convergence of the projection algorithm can be established by showing that the low-rank quaternion matrix manifold and the zero real component quaternion matrix manifold has a nontrivial intersection point. Numerical examples on synthetic pure quaternion matrices and color images are presented to illustrate the projection algorithm can find optimal low-rank pure quaternion approximation for pure quaternion matrices or color images.

Fast and Accurate Amplitude Demodulation of Wideband Signals

Amplitude demodulation is a classical operation used in signal processing. For a long time, its effective applications in practice have been limited to narrowband signals. In this work, we generalize amplitude demodulation to wideband signals. We pose demodulation as a recovery problem of an oversampled corrupted signal and introduce special iterative schemes belonging to the family of alternating projection algorithms to solve it. Sensibly chosen structural assumptions on the demodulation outputs allow us to reveal the high inferential accuracy of the method over a rich set of relevant signals. This new approach surpasses current state-of-the-art demodulation techniques apt to wideband signals in computational efficiency by up to many orders of magnitude with no sacrifice in quality. Such performance opens the door for applications of the amplitude demodulation procedure in new contexts. In particular, the new method makes online and large-scale offline data processing feasible, including the calculation of modulator-carrier pairs in higher dimensions and poor sampling conditions, independent of the signal bandwidth. We illustrate the utility and specifics of applications of the new method in practice by using natural speech and synthetic signals.

3-D Characterization of Urban Areas Using High-Resolution Polarimetric SAR Tomographic Techniques and a Minimal Number of Acquisitions

This article addresses the 3-D reconstruction of urban areas using a minimal number of Synthetic Aperture Radar (SAR) acquisitions, that is, a set of three images, characterized by intermediate spatial resolution features. In such extreme conditions, conventional tomographic techniques reveal unadapted to refined 3-D imaging purposes, either due to the resulting intrinsic coarse vertical resolution, or to the low dimensionality of the data set, that prevents any separation of complex mixed scattering patterns. A new high-resolution (HR) tomographic estimator, based on a polarimetric signal subspace fitting criterion, is proposed to overcome these limitations, as this method adapts to the statistical behavior of the backscattered signals using robust metrics. The optimization of the corresponding focusing criterion is led through a new polarimetric alternating projection algorithm, characterized by a low computational cost, which may also be used to optimize the polarimetric the deterministic maximum likelihood criterion. The proposed polarimetric signal subspace fitting technique is shown to outperform the other studied HR techniques over both simulated signals and data acquired by the DLR’s ESAR sensor at L-band over Dresden city, Germany. Finally full-rank polarimetric tomographic estimators are proposed that generalize nonparametric polarimetric estimators, and permit to estimate second-order polarimetric representations in 3-D, instead of unitary rank target vector with their conventional versions. This approach makes it possible to characterize polarimetric scattering mechanisms in 3-D.

Percolation of Estimates for $${{\bar{\partial }}}$$ by the Method of Alternating Projections

The method of alternating projections is used to examine how regularity of operators associated to the $${{\bar{\partial }}}$$ -Neumann problem percolates up the $${{\bar{\partial }}}$$ -complex. The approach revolves around operator identities—rather than estimates—that hold on any Lipschitz domain in $${{\mathbb {C}}}^n$$ , not necessarily bounded or pseudoconvex. We show that a geometric rate of convergence in von Neumann’s alternating projection algorithm, applied to two basic projection operators, is equivalent to $${{\bar{\partial }}}$$ having closed range. This implies that compactness of the $${{\bar{\partial }}}$$ -Neumann operator percolates up the $${{\bar{\partial }}}$$ -complex whenever $${{\bar{\partial }}}$$ has closed range at the corresponding form levels.

A novel Collaborative Filtering recommendation approach based on Soft Co-Clustering

Collaborative Filtering (CF) recommendation algorithm has been widely applied into recommender systems. Many CF algorithms associate a user/an item with one of subgroups by explicit or implicit features. However, considering that users may have multiple personalities and items may have diverse attributes, it is more reasonable to associate a user/an item with more than one group. In this paper, we propose the Soft K-indicators Alternative Projection (SKAP) algorithm, which can efficiently resolve soft clustering problem with high dimensions, to generate a sparse partition matrix and further a Top-N recommendation list is given. Unlike fuzzy C-means clustering, the SKAP algorithm is independent on the selection of initial values. In addition to that, we integrate the item type information into recommender systems to improve recommendation accuracy. Experimental results show that the proposed approach behaves superior performance in Top-N recommendation in terms of classical metrics and further show that multi-label classification framework is a better description than classical Co-Clustering framework.

A Minimization Approach for Constructing Generalized Barycentric Coordinates and Its Computation

We are interested in constructing more generalized barycentric coordinates (GBC) over arbitrary polygon in the 2D setting. We propose a constrained minimization over the class of infinitely differentiable functions subject to the GBC constraints of preserving linear functions and the non-negativity condition. It includes the harmonic GBC, biharmonic GBC, maximum entropy GBC, local barycentric coordinates as special cases. We mainly show that the constrained minimization has a unique solution when the minimizing functional is strictly convex. Next we use a $$C^r$$ smoothness spline function space $$S^r_d(\triangle )$$ with $$r\ge 2$$ over a triangulation $$\triangle $$ of a polygon of interest in $$\mathbb {R}^2$$ to approximate the minimizer. One advantage of using smooth splines is that derivaties, e.g. the mean curvature and/or Gaussian curvature of spline GBC functions can be calculated. As the minimization restricted to the spline space $$S^r_d$$ certainly has a unique minimizer, we use the standard projected gradient descent (PGD) method to approximate the spline minimizer. To find the projection of each iteration, we shall explain an alternating projection algorithm (APA). A convergence of the APA and the convergence of the PGD with the APA will be presented. As an example of this approach, a new kind of biharmonic GBC functions which preserve the nonnegativity is constructed. Finally, we have implemented the PGD method based on bivariate splines of arbitrary degree d and arbitrary smoothness r over arbitrary triangulation as long as $$d>>r$$ . The surfaces of many new GBC’s will be shown. Some standard GBC applications will be demonstrated.

Improved alternating projection algorithm for pattern synthesis with dual polarised conformal arrays

In this study, an improved alternating projection algorithm is presented for the pattern synthesis of a dual-polarised conformal array. An extra desired pattern is introduced to find an appropriate solution in the process of projecting the desired solution set to the feasible solution set. Virtual interferences are introduced to modify the solution iteratively based on the deviation between the obtained and desired pattern. Finally, the authors' algorithm is applied to dual-polarised conformal arrays and can achieve a suitable solution set between the predefined upper and lower bounds, which can synthesise both orthogonal polarisation components of the array pattern more effectively. Simulation examples are presented to verify the validity of the proposed algorithm.

Elasticity [formula omitted]-tensors and the strong ellipticity condition

In this paper, we establish two sufficient conditions for the strong ellipticity of any fourth-order elasticity tensor and investigate a class of tensors satisfying the strong ellipticity condition, the elasticity M-tensor. The first sufficient condition is that the strong ellipticity holds if the unfolding matrix of this fourth-order elasticity tensor can be modified into a positive definite one by preserving the summations of some corresponding entries. Second, an alternating projection algorithm is proposed to verify whether an elasticity tensor satisfies the first condition or not. Besides, the elasticity M-tensor is defined with respect to the M-eigenvalues of elasticity tensors. We prove that any nonsingular elasticity M-tensor satisfies the strong ellipticity condition by employing a Perron-Frobenius-type theorem for M-spectral radii of nonnegative elasticity tensors. Other equivalent definitions of nonsingular elasticity M-tensors are also established.

Trajectories from Distribution-valued Functional Curves: A Unified Wasserstein Framework★.

Temporal changes in medical images are often evaluated along a parametrized function that represents a structure of interest (e.g. white matter tracts). By attributing samples along these functions with distributions of image properties in the local neighborhood, we create distribution-valued signatures for these functions. We propose a novel and comprehensive framework which models their temporal evolution trajectories. This is achieved under the unifying scheme of Wasserstein distance metric. The regression problem is formulated as a constrained optimization problem and solved using an alternating projection algorithm. The solution simultaneously preserves the functional characteristics of the curve, models the temporal change in distribution profiles and forces the estimated distributions to be valid. Hypothesis testing is applied in two ways using Wasserstein based test statistics. Validation is presented on synthetic data. Detection of delayed growth is shown on DTI tracts, for a pediatric subject with respect to a healthy population of infants.

An alternating projection algorithm for the “approximate” GCD calculation

In the paper an approach is proposed for calculating the “best” approximate GCD of a set of coprime polynomials. The algorithm is motivated by the factorisation of the Sylvester resultant matrix of polynomial sets with nontrivial GCD. In the (generic) case of coprime polynomial sets considered here the aim is to minimise the norm of the residual error matrix of the inexact factorisation in order to compute the “optimal” approximate GCD. A least-squares alternating projection algorithm is proposed as an alternative to the solution of the corresponding optimisation problem via nonlinear programming techniques. The special structure of the problem in this case, however, means that the algorithm can be reduced to a sequence of standard subspace projections and hence no need arises to compute gradient vectors, Hessian matrices or optimal step-lengths. An estimate of the asymptotic convergence rate of the algorithm is finally established via the inclination of two subspaces.

Phase-Only Robust Minimum Dispersion Beamforming

A phase-only robust minimum dispersion (PO-RMD) beamformer is devised for non-Gaussian signals. Unlike conventional beamformers that adjust the complex-valued weights, including both amplitude and phase, of each antenna to fulfill spatial filtering, the proposed PO-RMD employs a unit-modulus constraint on the weights, which is equivalent to simply phase shifting at each antenna. Instead of the widely used minimum variance criterion, the PO-RMD adopts the minimum dispersion criterion, which minimizes the ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> -norm of the array output to utilize the non-Gaussianity of the signals. To achieve robustness against model uncertainty, the magnitude response of any steering vector within an uncertainty region is forced to exceed the threshold. A gradient projection (GP) algorithmic framework is developed to solve the resulting nonconvex optimization problem. In order to find a feasible point in the intersection of the unit-modulus, and robustness constraint sets in each iteration, an alternating projection algorithm is also devised. More importantly, the closed-form expressions of the projection onto the two sets are derived, which only need a low complexity of O(M) with M being the number of antennas. Convergence of the alternating projection is analyzed, and the local linear convergence rate is established. Simulation results demonstrate the fast convergence rate, and accuracy of the GP, as well as the simplicity and robustness of the PO-RMD.

A linearly convergent doubly stochastic Gauss–Seidel algorithm for solving linear equations and a certain class of over-parameterized optimization problems

Consider the classical problem of solving a general linear system of equations $Ax=b$. It is well known that the (successively over relaxed) Gauss-Seidel scheme and many of its variants may not converge when $A$ is neither diagonally dominant nor symmetric positive definite. Can we have a linearly convergent G-S type algorithm that works for {\it any} $A$? In this paper we answer this question affirmatively by proposing a doubly stochastic G-S algorithm that is provably linearly convergent (in the mean square error sense) for any feasible linear system of equations. The key in the algorithm design is to introduce a {\it nonuniform double stochastic} scheme for picking the equation and the variable in each update step as well as a stepsize rule. These techniques also generalize to certain iterative alternating projection algorithms for solving the linear feasibility problem $A x\le b$ with an arbitrary $A$, as well as high-dimensional minimization problems for training over-parameterized models in machine learning. Our results demonstrate that a carefully designed randomization scheme can make an otherwise divergent G-S algorithm converge.

Extrapolative phase retrieval based on a hybrid of PhaseCut and alternating projection techniques

Phase retrieval is an important research topic in the optics field. It is aimed to reconstruct the wave-front from the intensity of the far-field diffracted field. To deal with the local stagnation of conventional iterative Fourier-based algorithms, an interesting recent approach formulates the phase retrieval problem as a tractable relaxation (called PhaseCut). Despite its powerful global capability, the operation of recasting a vector problem into a matrix problem leads to higher computational cost, which limits its use in far-field patterns with high resolution. In this paper, we develop an extrapolative alternating projection algorithm to cope with the reconstruction of high-resolution phase pattern. The algorithm starts with a low-resolution phase estimation solved by PhaseCut from the far-field amplitudes in the low frequency range, and then refines the initial estimation by an extrapolative iterative projection method, which has a low computational complexity. The iterative process is divided into various periods where the inputs of the algorithm (far-field amplitudes) are extrapolated to include more high frequency information. The main contribution is that the algorithm can quickly converge to the globally optimal and high-resolution solution with a high probability. We verified the validity of the algorithm with various numerical simulations.

A Krasnoselskii-Mann Algorithm With an Improved EM Preconditioner for PET Image Reconstruction.

This paper presents a preconditioned Krasnoselskii-Mann (KM) algorithm with an improved EM preconditioner (IEM-PKMA) for higher-order total variation (HOTV) regularized positron emission tomography (PET) image reconstruction. The PET reconstruction problem can be formulated as a three-term convex optimization model consisting of the Kullback-Leibler (KL) fidelity term, a nonsmooth penalty term, and a nonnegative constraint term which is also nonsmooth. We develop an efficient KM algorithm for solving this optimization problem based on a fixed-point characterization of its solution, with a preconditioner and a momentum technique for accelerating convergence. By combining the EM precondtioner, a thresholding, and a good inexpensive estimate of the solution, we propose an improved EM preconditioner that can not only accelerate convergence but also avoid the reconstructed image being "stuck at zero." Numerical results in this paper show that the proposed IEM-PKMA outperforms existing state-of-the-art algorithms including, the optimization transfer descent algorithm and the preconditioned L-BFGS-B algorithm for the differentiable smoothed anisotropic total variation regularized model, the preconditioned alternating projection algorithm, and the alternating direction method of multipliers for the nondifferentiable HOTV regularized model. Encouraging initial experiments using clinical data are presented.

Infimal convolution-based regularization for SPECT reconstruction.

Total variation (TV) regularization is efficient in suppressing noise, but is known to suffer from staircase artifacts. The goal of this work was to develop a regularization method using the infimal convolution of the first- and the second-order derivatives to reduce or even prevent staircase artifacts in the reconstructed images, and to investigate if the advantage in noise suppression by this TV-type regularization can be translated into dose reduction. In the present work, we introduce the infimal convolution of the first- and the second-order total variation (ICTV) as the regularization term in penalized maximum likelihood reconstruction. The preconditioned alternating projection algorithm (PAPA), previously developed by the authors of this article, was employed to produce the reconstruction. Using Monte Carlo-simulated data, we evaluate noise properties and lesion detectability in the reconstructed images and compare the results with conventional total variation (TV) and clinical EM-based methods with Gaussian post filter (GPF-EM). We also evaluate the quality of ICTV regularized images obtained for lower photon number data, compared with clinically used photon number, to verify the feasibility of radiation-dose reduction to patients by use of the ICTV reconstruction method. By comparison with GPF-EM reconstructed images, we have found that the ICTV-PAPA method can achieve a lower background variability level while maintaining the same level of contrast. Images reconstructed by the ICTV-PAPA method with 80,000 counts per view exhibit even higher channelized Hotelling observer (CHO) signal-to-noise ratio (SNR), as compared to images reconstructed by the GPF-EM method with 120,000 counts per view. In contrast to the TV-PAPA method, the ICTV-PAPA reconstruction method avoids substantial staircase artifacts, while producing reconstructed images with higher CHO SNR and comparable local spatial resolution. Simulation studies indicate that a 33% dose reduction is feasible by switching to the ICTV-PAPA method, compared with the GPF-EM clinical standard.

Discrete Spectral Hashing for Efficient Similarity Retrieval.

To meet the required huge data analysis, organization, and storage demand, the hashing technique has got a lot of attention as it aims to learn an efficient binary representation from the original high-dimensional data. In this paper, we focus on the unsupervised spectral hashing due to its effective manifold embedding. Existing spectral hashing methods mainly suffer from two problems, i.e., the inefficient spectral candidate and intractable binary constraint for spectral analysis. To overcome these two problems, we propose to employ spectral rotation to seek a better spectral solution and adopt the alternating projection algorithm to settle the complex code constraints, which are therefore named as Spectral Hashing with Spectral Rotation and Alternating Discrete Spectral Hashing, respectively. To enjoy the merits of both methods, the spectral rotation technique is finally combined with the original spectral objective, which aims to simultaneously learn better spectral solution and more efficient discrete codes and is called as Discrete Spectral Hashing. Furthermore, the efficient optimization algorithms are also provided, which just take comparable time complexity to existing hashing methods. To evaluate the proposed three methods, extensive comparison experiments and studies are conducted on four large-scale data sets for the image retrieval task, and the noticeable performance beats several state-of-the-art spectral hashing methods on different evaluation metrics.

Alternating projection method for a class of tensor equations

This paper considers how to solve a class of tensor equations arising from the unified definition of tensor–vector products. Of special interest is the order-3 tensor equation whose solutions are the intersection of a group of quadrics from a geometric point of view. Inspired by the method of alternating projections for set intersection problems, we develop a hybrid alternating projection algorithm for solving order-3 tensor equations. The local linear convergence of the alternating projection method is established under suitable conditions. Some numerical experiments are conducted to evaluate the effect of the proposed algorithm.

Modeling Persistent Trends in Distributions

ABSTRACTWe present a nonparametric framework to model a short sequence of probability distributions that vary both due to underlying effects of sequential progression and confounding noise. To distinguish between these two types of variation and estimate the sequential-progression effects, our approach leverages an assumption that these effects follow a persistent trend. This work is motivated by the recent rise of single-cell RNA-sequencing experiments over a brief time course, which aim to identify genes relevant to the progression of a particular biological process across diverse cell populations. While classical statistical tools focus on scalar-response regression or order-agnostic differences between distributions, it is desirable in this setting to consider both the full distributions as well as the structure imposed by their ordering. We introduce a new regression model for ordinal covariates where responses are univariate distributions and the underlying relationship reflects consistent changes in the distributions over increasing levels of the covariate. This concept is formalized as a trend in distributions, which we define as an evolution that is linear under the Wasserstein metric. Implemented via a fast alternating projections algorithm, our method exhibits numerous strengths in simulations and analyses of single-cell gene expression data. Supplementary materials for this article are available online.

Linear convergence of the generalized Douglas–Rachford algorithm for feasibility problems

In this paper, we study the generalized Douglas-Rachford algorithm and its cyclic variants which include many projection-type methods such as the classical Douglas-Rachford algorithm and the alternating projection algorithm. Specifically, we establish several local linear convergence results for the algorithm in solving feasibility problems with finitely many closed possibly nonconvex sets under different assumptions. Our findings not only relax some regularity conditions but also improve linear convergence rates in the literature. In the presence of convexity, the linear convergence is global.

MAP-MRF-Based Super-Resolution Reconstruction Approach for Coded Aperture Compressive Temporal Imaging

Coded Aperture Compressive Temporal Imaging (CACTI) can afford low-cost temporal super-resolution (SR), but limits are imposed by noise and compression ratio on reconstruction quality. To utilize inter-frame redundant information from multiple observations and sparsity in multi-transform domains, a robust reconstruction approach based on maximum a posteriori probability and Markov random field (MAP-MRF) model for CACTI is proposed. The proposed approach adopts a weighted 3D neighbor system (WNS) and the coordinate descent method to perform joint estimation of model parameters, to achieve the robust super-resolution reconstruction. The proposed multi-reconstruction algorithm considers both total variation (TV) and ℓ 2 , 1 norm in wavelet domain to address the minimization problem for compressive sensing, and solves it using an accelerated generalized alternating projection algorithm. The weighting coefficient for different regularizations and frames is resolved by the motion characteristics of pixels. The proposed approach can provide high visual quality in the foreground and background of a scene simultaneously and enhance the fidelity of the reconstruction results. Simulation results have verified the efficacy of our new optimization framework and the proposed reconstruction approach.