Alternating Projection Method Research Articles

Designing incoherent unit norm tight frames via block coordinate descent-based alternating projection

In this paper, we introduce a novel methodology for constructing incoherent unit norm tight frames, termed block coordinate descent-based alternating projection. In the proposed approach, the underlying min-max problem was reformulated in a block-wise fashion by taking a fixed number of columns as a block and adding a spectral constraint, and it was approximately solved via an alternating projection method with Nesterov accelerated gradient technique. Numerical experiments demonstrate that the resulting algorithm is particularly efficient for the construction of incoherent unit norm tight frames with high redundancy and has great advantages in running efficiency.

Alternating projection combined with fast gradient projection (FGP-AP) method for intensity-only measurement optical diffraction tomography in LED array microscopy.

Optical diffraction tomography (ODT) is a powerful label-free measurement tool that can quantitatively image the three-dimensional (3D) refractive index (RI) distribution of samples. However, the inherent "missing cone problem," limited illumination angles, and dependence on intensity-only measurements in a simplified imaging setup can all lead to insufficient information mapping in the Fourier domain, affecting 3D reconstruction results. In this paper, we propose the alternating projection combined with the fast gradient projection (FGP-AP) method to compensate for the above problem, which effectively reconstructs the 3D RI distribution of samples using intensity-only images captured from LED array microscopy. The FGP-AP method employs the alternating projection (AP) algorithm for gradient descent and the fast gradient projection (FGP) algorithm for regularization constraints. This approach is equivalent to incorporating prior knowledge of sample non-negativity and smoothness into the 3D reconstruction process. Simulations demonstrate that the FGP-AP method improves reconstruction quality compared to the original AP method, particularly in the presence of noise. Experimental results, obtained from mouse kidney cells and label-free blood cells, further affirm the superior 3D imaging efficacy of the FGP-AP method.

Localization of brain signals by alternating projection

A popular approach for modeling brain activity in MEG and EEG is based on a small set of current dipoles, where each dipole represents the combined activation of a local area of the brain. Here, we address the problem of multiple dipole localization with a novel solution called Alternating Projection (AP). The AP solution is based on minimizing the least-squares (LS) criterion by transforming the multi-dimensional optimization required for direct LS solution, to a sequential and iterative solution in which one source at a time is localized, while keeping the other sources fixed. Results from simulated, phantom, and human MEG data demonstrated the high accuracy of the AP method, with superior localization results than popular scanning methods from the multiple-signal classification (MUSIC) and beamformer families. In addition, the AP method was more robust to forward model errors resulting from head rotations and translations, as well as different cortex tessellation grids for the forward and inverse solutions, with consistently higher localization accuracy in low SNR and highly correlated sources.

A hybrid direct search and projected simplex gradient method for convex constrained minimization

We propose a new Derivative-free Optimization (DFO) approach for solving convex constrained minimization problems. The feasible set is assumed to be the non-empty intersection of a finite collection of closed convex sets, such that the projection onto each of these individual convex sets is simple and inexpensive to compute. Our iterative approach alternates between steps that use Directional Direct Search (DDS), considering adequate poll directions, and a Spectral Projected Gradient (SPG) method, replacing the real gradient by a simplex gradient, under a DFO approach. In the SPG steps, if the convex feasible set is simple, then a direct projection is computed. If the feasible set is the intersection of finitely many convex simple sets, then Dykstra's alternating projection method is applied. Convergence properties are established under standard assumptions usually associated to DFO methods. Some preliminary numerical experiments are reported to illustrate the performance of the proposed algorithm, in particular by comparing it with a classical DDS method. Our results indicate that the hybrid algorithm is a robust and effective approach for derivative-free convex constrained optimization.

Tangent Space Based Alternating Projections for Nonnegative Low Rank Matrix Approximation

In this paper, we develop a new alternating projection method to compute nonnegative low rank matrix approximation for nonnegative matrices. In the nonnegative low rank matrix approximation method, the projection onto the manifold of fixed rank matrices can be expensive as the singular value decomposition is required. We propose to use the tangent space of the point in the manifold to approximate the projection onto the manifold in order to reduce the computational cost. We show that the sequence generated by the alternating projections onto the tangent spaces of the fixed rank matrices manifold and the nonnegative matrix manifold, converge linearly to a point in the intersection of the two manifolds where the convergent point is sufficiently close to optimal solutions. This convergence result based inexact projection onto the manifold is new and is not studied in the literature. Numerical examples in data clustering, pattern recognition and hyperspectral data analysis are given to demonstrate that the performance of the proposed method is better than that of nonnegative matrix factorization methods in terms of computational time and accuracy.

TwP: Two-stage projection framework with manifold constraint for image reconstruction

As an extensively studied task in computer vision, image reconstruction aims to reconstruct original data from acquired measurements. It is a highly ill-posedness inverse problem since a countably infinite number of feasible solutions exist. To achieve high-quality reconstruction performance, we propose a two-stage projection (TwP) framework built upon the projection technique and manifold constraint. We first exploit the generalized non-convex alternating projection method for image reconstruction in the pre-reconstruction stage, that incorporates the image manifold constraint implicitly imposed by deep denoiser. It provides a unified method to solve linear/non-linear inverse problems in a plug-and-play manner and deliver a good estimation. In the post-processing stage, a feed-forward network fine-tuned for a specific task is utilized to reproject the pre-reconstruction results onto the low-dimensional image manifold, which enables the outcomes to be in or closest to the manifold. Extensive experiments across various image reconstruction tasks, such as single-photon imaging, image compressive sensing, and image deblurring, demonstrate that the proposed TwP delivers promising results against some existing state-of-the-art algorithms in terms of objective and visual perceptions.

Sketching for a low-rank nonnegative matrix approximation: Numerical study

Abstract We propose new approximate alternating projection methods, based on randomized sketching, for the low-rank nonnegative matrix approximation problem: find a low-rank approximation of a nonnegative matrix that is nonnegative, but whose factors can be arbitrary. We calculate the computational complexities of the proposed methods and evaluate their performance in numerical experiments. The comparison with the known deterministic alternating projection methods shows that the randomized approaches are faster and exhibit similar convergence properties.

Optimal iterative learning control design for continuous-time systems with nonidentical trial lengths using alternating projections between multiple sets

Iterative learning control (ILC) applies to systems that repeat the same finite duration task repeatedly. Each repetition is usually termed as a trial, and the associated duration is called the trial length. Once a trial is completed, all information is available for use in updating the control input for the subsequent trial. The vast majority of the currently available designs demand a strictly identical trial length. This paper gives a new result on the design and analysis for continuous-time linear dynamics based on a modified alternating projection method, where the trial lengths may be nonidentical. This result employs multiple sets to represent the actual varying trial length dynamics and is developed by reformulating the problem to one that minimizes the defined distance in a Hilbert space setting. Compared to the standard alternating projections using two sets, the theory of alternating projections between multiple sets is employed to obtain deterministic convergence result for the nonidentical trial length problem. A numerical case study is also given to illustrate the application of the new design.

ALTERNATING PROJECTION METHOD FOR SOLVING DOUBLY STOCHASTIC INVERSE SINGULAR VALUE PROBLEMS WITH PRESCRIBED ENTRIES

Doubly stochastic inverse singular value problem with prescribed entries aims to construct a doubly stochastic matrix from the prescribed singular value and prescribed entries. In this paper, the doubly stochastic inverse singular value problem is considered as the problem of finding a point in the intersection of a compact set and a closed convex set. We present a numerical procedure which is based on an alternating projection process, for solving the problem. The method is iterative in nature. And each subproblem in the alternating projection method can be solved easily. Convergence properties of the algorithm are investigated and numerical results are presented to illustrate the effectiveness of our method.

Phase Estimation for Distributed Scatterers by Alternating Projection

Decorrelation is a major obstacle to the application of multitemporal interferometric synthetic aperture radar (InSAR) in areas with low coherence. Distributed scatterers (DSs) with similar backscattering in the spatial neighborhood are the key to improving the observational density and accuracy of deformation estimation in fast decorrelation regions. Phase series estimation from all possible interferograms is expected to improve the signal-to-noise ratio (SNR) and further enhance the sensitivity of deformation measurement. The coherence bias and the efficiency of the phase estimation raise concerns. In this article, we propose a computationally attractive algorithm for the interferometric phase estimation, namely alternating projection (AP), which is a combination of the alternating maximization and the projection matrix decomposition methods. The homogenous pixel selection and coherence estimation bias correction are conducted by the FaSHPS algorithm and DSIpro software toolbox. Results of simulations and real SAR data show that the proposed AP method could reconstruct credible phase series comparable to the quasi-Newton optimization algorithm [Broyden–Fletcher–Goldfarb–Shanno (BFGS)] while having three times the efficiency gain.

Alternating projection-based phase optimization for arbitrary glare suppression through multimode fiber

Wavefront shaping for glare suppression helps to reduce the speckle background through scattering media and enables imaging, sensing, and some speckle-related advanced applications through customizing the speckle light field. For glare suppression in a target region, current methods are either slow or not sufficiently generic. Here, an alternating projection method that fully exploits the transmission matrix of the scattering medium is proposed for fast and arbitrary glare suppression. Parallelly, multiple phase masks corresponding to different target regions can be computationally optimized without iterative hardware feedback. Numerical simulation shows that under phase-only modulation, a suppression factor of ∼10−3 can be realized for a large target region with only 30 iterations. With the use of a graphics processing unit and a digital micromirror device, fast and effective glare suppression for target regions of various shapes and sizes was demonstrated at the distal end of a multimode fiber. This technique could be promising for applications like multimode fiber-based speckle optical tweezer or endoscopy with speckle illumination inside complex environments.

An efficient hybrid method for fast phased array pattern synthesizing

A novel and fast array synthesis method simultaneously combining the advantages of genetic algorithm (GA), alternating projection method (APM), and differential evolution (DE) is proposed in this work. The APM adopts fast Fourier transform (FFT), and the far-field calculation is mathematically simplified to multiplication with a common matrix, which could be faster than inverted-FFT (IFFT) for thousands of individuals of the same small array. Then an effective fitness evaluating method with multi-objective is developed, which could be adjusted according to specific needs such as array side lobe suppression, beam-broaden, and beam-shaping. To demonstrate the feasibility of the proposed method, several array synthesis examples are presented and the results show that the proposed method has good robustness, convergence, and global searching ability.

A penalized method of alternating projections for weighted low-rank hankel matrix optimization

Weighted low-rank Hankel matrix optimization has long been used to reconstruct contaminated signal or forecast missing values for time series of a wide class. The Method of Alternating Projections (MAP) (i.e., alternatively projecting to a low-rank matrix manifold and the Hankel matrix subspace) is a leading method. Despite its wide use, MAP has long been criticized of lacking convergence and of ignoring the weights used to reflect importance of the observed data. The most of known results are in a local sense. In particular, the latest research shows that MAP may converge at a linear rate provided that the initial point is close enough to a true solution and a transversality condition is satisfied. In this paper, we propose a globalized variant of MAP through a penalty approach. The proposed method inherits the favourable local properties of MAP and has the same computational complexity. Moreover, it is capable of handling a general weight matrix, is globally convergent, and enjoys local linear convergence rate provided that the cutting off singular values are significantly smaller than the kept ones. Furthermore, the new method also applies to complex data. Extensive numerical experiments demonstrate the efficiency of the proposed method against several popular variants of MAP.

Circumcentering approximate reflections for solving the convex feasibility problem

The circumcentered-reflection method (CRM) has been applied for solving convex feasibility problems. CRM iterates by computing a circumcenter upon a composition of reflections with respect to convex sets. Since reflections are based on exact projections, their computation might be costly. In this regard, we introduce the circumcentered approximate-reflection method (CARM), whose reflections rely on outer-approximate projections. The appeal of CARM is that, in rather general situations, the approximate projections we employ are available under low computational cost. We derive convergence of CARM and linear convergence under an error bound condition. We also present successful theoretical and numerical comparisons of CARM to the original CRM, to the classical method of alternating projections (MAP), and to a correspondent outer-approximate version of MAP, referred to as MAAP. Along with our results and numerical experiments, we present a couple of illustrative examples.

Application of Aperiodic Tiling to Lattice Spatial Structures

The aim of this report is to expand on the existing available configurations for lattice spa- tial structures and in turn widen the range of design options that can be presented to architects and/or clients. This is achieved by developing a new family of configurations in which the aperiodic tiling pattern ‘Penrose Kite and Dart’ tiling is applied to spherical caps using formex configuration pro- cessing techniques. To further expand this new family, the pattern modification of triangulation is suggested along with alternative projection methods of changing the shape of the initial tiling patch as well as using pellevation. To determine a degree of feasibility, the regularity of components is stud- ied and compared to configurations of existing, periodic patterns such as ribbed and diamatic domes. As a result, the spherical caps under two decompositions of aperiodic tiling were found to have simi- lar regularity indicators to the periodic configurations and are therefore considered feasible options. Each proposed modification was also found to be low impacting on feasibility by introducing a low number of new member lengths. However, the use of aperiodic tiling under three or more decomposi- tions is not recommended due to the significant increase in new member lengths.

Alternating conditional gradient method for convex feasibility problems

The classical convex feasibility problem in a finite dimensional Euclidean space consists of finding a point in the intersection of two convex sets. In the present paper we are interested in two particular instances of this problem. First, we assume to know how to compute an exact projection onto one of the sets involved and the other set is compact such that the conditional gradient (CondG) method can be used for computing efficiently an inexact projection on it. Second, we assume that both sets involved are compact such that the CondG method can be used for computing efficiently inexact projections on them. We combine alternating projection method with CondG method to design a new method, which can be seen as an inexact feasible version of alternate projection method. The proposed method generates two different sequences belonging to each involved set, which converge to a point in the intersection of them whenever it is not empty. If the intersection is empty, then the sequences converge to points in the respective sets whose distance between them is equal to the distance between the sets in consideration. Numerical experiments are provided to illustrate the practical behavior of the method.

Least-squares support tensor data description

Multi-way arrays or tensors are becoming important tools to deal with multidimensional data. This paper is concerned with a one-class classification of second order tensor data. One-class classification problems arise when data from one class is the only data available. Typical vector based methods have limitations when dealing with tensor data. Least squares one-class support vector machine (LS-OCSVM) is a variant of the standard one-class support vector machine (OCSVM). It operates directly on patterns represented by vector and obtains an analytical solution directly from solving a set of linear equations instead of quadratic programming (QP). One-class support tensor machine (OCSTM) is a one-class classification tool for solving the one-class classification problem for tensors. It is based on alternating projection algorithms and quadratic programming to obtain solutions. In this paper, a new method to deal with one-classification of tensors is proposed, least squares support tensor data description (LS-STDD). The advantage of LS-STDD over one-class support tensor machine (OCSTM) is that LS-STDD has a closed form solution. It doesn’t use the alternating projection method of OCSTM and quadratic programming. Consequently, LS-STDD is easier and faster to implement than OCSTM. The efficiency of the proposed method over OCSTM is illustrated through simulations.

A note on the finite convergence of alternating projections

We establish sufficient conditions for finite convergence of the alternating projections method for two non-intersecting and potentially nonconvex sets. Our results are based on a generalization of the concept of intrinsic transversality, which until now has been restricted to sets with nonempty intersection. In the special case of a polyhedron and closed half space, our sufficient conditions define the minimum distance between the two sets that is required for alternating projections to converge in a single iteration.

A variational approach to the alternating projections method

The 2-sets convex feasibility problem aims at finding a point in the nonempty intersection of two closed convex sets A and B in a Hilbert space H. The method of alternating projections is the simplest iterative procedure for finding a solution and it goes back to von Neumann. In the present paper, we study some stability properties for this method in the following sense: we consider two sequences of closed convex sets {A_n} and {B_n}, each of them converging, with respect to the Attouch-Wets variational convergence, respectively, to A and B. Given a starting point a_0, we consider the sequences of points obtained by projecting on the “perturbed” sets, i.e., the sequences {a_n} and {b_n} given by b_n=P_{B_n}(a_{n-1}) and a_n=P_{A_n}(b_n). Under appropriate geometrical and topological assumptions on the intersection of the limit sets, we ensure that the sequences {a_n} and {b_n} converge in norm to a point in the intersection of A and B. In particular, we consider both when the intersection Acap B reduces to a singleton and when the interior of A cap B is nonempty. Finally we consider the case in which the limit sets A and B are subspaces.

The circumcentered-reflection method achieves better rates than alternating projections

We study the convergence rate of the Circumcentered-Reflection Method (CRM) for solving the convex feasibility problem and compare it with the Method of Alternating Projections (MAP). Under an error bound assumption, we prove that both methods converge linearly, with asymptotic constants depending on a parameter of the error bound, and that the one derived for CRM is strictly better than the one for MAP. Next, we analyze two classes of fairly generic examples. In the first one, the angle between the convex sets approaches zero near the intersection, so that the MAP sequence converges sublinearly, but CRM still enjoys linear convergence. In the second class of examples, the angle between the sets does not vanish and MAP exhibits its standard behavior, i.e., it converges linearly, yet, perhaps surprisingly, CRM attains superlinear convergence.