Riemannian Conjugate Gradient Research Articles

Riemannian Optimal Identification Method for Linear Systems With Symmetric Positive-Definite Matrix

This article develops identification methods for linear continuous-time symmetric systems, such as electrical network systems, multiagent network systems, and temperature dynamics in buildings. To this end, we formulate three system identification problems for the corresponding discrete-time systems. The first is a least-squares problem in which we wish to minimize the sum of squared errors between the true and model outputs on the product manifold of the manifold of symmetric positive-definite matrices and two Euclidean spaces. In the second problem, to reduce the search dimensions, the product manifold is replaced with the quotient set under a specified group action by the orthogonal group. In the third problem, the manifold of symmetric positive-definite matrices in the first problem is replaced by the manifold of matrices with only positive diagonal elements. In particular, we examine the quotient geometry in the second problem. We propose Riemannian conjugate gradient methods for the three problems, and select initial points using a popular subspace method. The effectiveness of our proposed methods is demonstrated through numerical simulations and comparisons with the Gauss–Newton method, which is one of the most popular approach for solving least-squares problems.

Wireless Power Transfer by Beamspace Large-Scale MIMO With Lens Antenna Array

In this paper, we study wireless power transfer (WPT) using the discrete lens array-based beamspace large-scale multiple-input multiple-output (MIMO) system. The channel matrix of beamspace MIMO exhibits sparse property, which enables the transmitter to employ only a small number of active antennas while maintaining the full MIMO performance. Hence, the number of radio frequency (RF) chains can be significantly reduced, cutting the cost of hardware implementation and circuit power consumption. We consider two WPT design problems in the beamspace MIMO system with constraints on the number of RF chains: the sum power transfer and the max–min power transfer problems; and for each problem, we consider both multi-stream and uni-stream transmissions. For the sum power transfer problem, we show that the uni-stream power transfer achieves the same performance as the multi-stream case, and we propose two algorithms for the uni-stream transmission, namely, an eigendecomposition-based greedy algorithm and a truncated power iteration algorithm. For the max–min power transfer problem, we propose a semidefinite relaxation-based greedy algorithm for the multi-stream power transfer and a Riemannian conjugate gradient algorithm for the uni-stream case. The simulation results show that with a small number of RF chains, the beamspace MIMO system significantly outperforms the conventional MIMO system in terms of WPT efficiency.

Matrix Factorization for Collaborative Budget Allocation

This paper studies the collaborative budget allocation problem in which users are not isolated in the collaborative consumption of goods or services when available goods or services are limited. Different from existing methods that treat each user independently, we investigate the geometric properties of user’s consumption or preference on services, and design a matrix completion framework on the simplex. In this framework, an item’s allocation vector indicating how available services are allocated to users is estimated by the combination of user profiles as basis points on the simplex. Instead of using Euclidean distance directly, we specify a Riemannian distance on the simplex or project histogram data on simplex to Euclidean space. To intensify our model’s stability, we relax the exact recovery constraint to make a robust collaborative prediction. The resulting objective function is then efficiently optimized by a Riemannian conjugate gradient method on the simplex. Experiments on real-world data sets demonstrate our model’s competitiveness versus other collaborative budget prediction methods. Comparisons of different distance metrics for histogram data are shown and discussed. Note to Practitioners —This paper was motivated by the collaborative budget allocation (CBA) problem in which either a user has limited ratings that can be distributed among items, or a service has limited availability to all users in the sharing economy system. Existing approaches rarely consider this phenomenon in their collaborative prediction modeling, which implies the geometric constraints on a user’s rating vector. This paper suggests a matrix factorization framework to address this problem by learning a group of a user profile as basis points that can be combined to recover other users’ rating vectors. Considering the noise in observed ratings in real applications, we enhance the robustness of collaborative prediction by relaxing the recovery constraint. We then propose to specify the distance metric on the simplex from two perspectives, and give the optimization approach. In the future research, we will address the CBA in more complex settings, for example, some items can be substitutes or complementary for each other, in this way, how items can be better distributed among users is an interesting problem.

Joint Multicast and Unicast Beamforming for Coded Caching

In this paper, we consider a multicell system with cache-equipped base stations (BSs) and users. A coded caching scheme is employed at user ends such that multiple requested file contents can be combined and encoded together at the BS, and decoded successfully at the users. The BSs can then multicast the coded cached content and unicast the uncached content to users. Two different system models are proposed based on different levels of BS cooperation. We formulate the joint multicast and unicast beamformer design problem to maximize the weighted user fairness rate subject to the per BS power constraint. We propose a Riemannian conjugate gradient algorithm to solve the problem, which operates on the multi-sphere manifold of the parameter space and has a low computational complexity but better performance compared with the existing semi-definite relaxation methods. Extensive simulation results are provided to demonstrate the performance of the proposed systems.

Estimating the square root of probability density function on Riemannian manifold

AbstractWe propose that the square root of a probability density function can be represented as a linear combination of Gaussian kernels. It is shown that if the Gaussian kernel centres and kernel width are known, then the maximum likelihood parameter estimator can be formulated as a Riemannian optimisation problem on sphere manifold. The first order Riemannian geometry of the sphere manifold and vector transport are initially explored, then the well‐known Riemannian conjugate gradient algorithm is used to estimate the model parameters. For completeness, the k‐means clustering algorithm and a grid search are employed to determine the centres and kernel width, respectively. Simulated examples are employed to demonstrate that the proposed approach is effective in constructing the estimate of the square root of probability density function.

Riemannian Optimal System Identification Algorithm for Linear MIMO Systems

In this letter, a novel optimization method for the data-driven local coordinates approach to a prediction error method, which is one of system identification methods, is developed. In the proposed method, parameters of system matrices to be estimated are kept as their original forms of matrices whereas they are converted into a vector in the literature. Optimization with respect to the matrix variables can improve the computational efficiency. Furthermore, it is effective to equate input–output equivalent systems with each other since they attain the same value of the objective function. To this end, a Riemannian quotient manifold is introduced. The geometry of the proposed manifold is investigated and a novel Riemannian conjugate gradient method on the manifold is provided. Numerical experiments show that the difference of the proposed Riemannian conjugate gradient method from Euclidean one greatly improves the efficiency of the algorithm.

Discriminative Structured Dictionary Learning on Grassmann Manifolds and Its Application on Image Restoration.

Image restoration is a difficult and challenging problem in various imaging applications. However, despite of the benefits of a single overcomplete dictionary, there are still several challenges for capturing the geometric structure of image of interest. To more accurately represent the local structures of the underlying signals, we propose a new problem formulation for sparse representation with block-orthogonal constraint. There are three contributions. First, a framework for discriminative structured dictionary learning is proposed, which leads to a smooth manifold structure and quotient search spaces. Second, an alternating minimization scheme is proposed after taking both the cost function and the constraints into account. This is achieved by iteratively alternating between updating the block structure of the dictionary defined on Grassmann manifold and sparsifying the dictionary atoms automatically. Third, Riemannian conjugate gradient is considered to track local subspaces efficiently with a convergence guarantee. Extensive experiments on various datasets demonstrate that the proposed method outperforms the state-of-the-art methods on the removal of mixed Gaussian-impulse noise.

Coordinated Multicell Multicast Beamforming Based on Manifold Optimization

Multicast beamforming is a key technology for next-generation wireless cellular networks to support high-rate content distribution services. In this paper, the coordinated downlink multicast beamforming design in multicell networks is considered. The goal is to maximize the minimum signal-to-interference-plus-noise ratio of all users under individual base station power constraints. We exploit the fractional form of the objective function and geometric properties of the con-straints to reformulate the problem as a parametric manifold optimization program. Afterwards we propose a low-complexity Dinkelbach-type algorithm combined with adaptive exponential smoothing and Riemannian conjugate gradient iteration, which is guaranteed to converge. Numerical experiments show that the proposed algorithm outperforms the existing SDP-based method and DC-programming-based method and achieves near-optimal performance.

Computation of Ground States of the Gross--Pitaevskii Functional via Riemannian Optimization

In this paper we combine concepts from Riemannian Optimization and the theory of Sobolev gradients to derive a new conjugate gradient method for direct minimization of the Gross-Pitaevskii energy functional with rotation. The conservation of the number of particles constrains the minimizers to lie on a manifold corresponding to the unit $L^2$ norm. The idea developed here is to transform the original constrained optimization problem to an unconstrained problem on this (spherical) Riemannian manifold, so that fast minimization algorithms can be applied as alternatives to more standard constrained formulations. First, we obtain Sobolev gradients using an equivalent definition of an $H^1$ inner product which takes into account rotation. Then, the Riemannian gradient (RG) steepest descent method is derived based on projected gradients and retraction of an intermediate solution back to the constraint manifold. Finally, we use the concept of the Riemannian vector transport to propose a Riemannian conjugate gradient (RCG) method for this problem. It is derived at the continuous level based on the "optimize-then-discretize" paradigm instead of the usual "discretize-then-optimize" approach, as this ensures robustness of the method when adaptive mesh refinement is performed in computations. We evaluate various design choices inherent in the formulation of the method and conclude with recommendations concerning selection of the best options. Numerical tests demonstrate that the proposed RCG method outperforms the simple gradient descent (RG) method in terms of rate of convergence. While on simple problems a Newton-type method implemented in the {\tt Ipopt} library exhibits a faster convergence than the (RCG) approach, the two methods perform similarly on more complex problems requiring the use of mesh adaptation. At the same time the (RCG) approach has far fewer tunable parameters.

A Riemannian conjugate gradient method for optimization on the Stiefel manifold

In this paper we propose a new Riemannian conjugate gradient method for optimization on the Stiefel manifold. We introduce two novel vector transports associated with the retraction constructed by the Cayley transform. Both of them satisfy the Ring-Wirth nonexpansive condition, which is fundamental for convergence analysis of Riemannian conjugate gradient methods, and one of them is also isometric. It is known that the Ring-Wirth nonexpansive condition does not hold for traditional vector transports as the differentiated retractions of QR and polar decompositions. Practical formulae of the new vector transports for low-rank matrices are obtained. Dai's nonmonotone conjugate gradient method is generalized to the Riemannian case and global convergence of the new algorithm is established under standard assumptions. Numerical results on a variety of low-rank test problems demonstrate the effectiveness of the new method.

Fast Hybrid Algorithm for Big Matrix Recovery

Large-scale Nuclear Norm penalized Least Square problem (NNLS) is frequently encountered in estimation of low rank structures. In this paper we accelerate the solution procedure by combining non-smooth convex optimization with smooth Riemannian method. Our methods comprise of two phases. In the first phase, we use Alternating Direction Method of Multipliers (ADMM) both to identify the fix rank manifold where an optimum resides and to provide an initializer for the subsequent refinement. In the second phase, two superlinearly convergent Riemannian methods: Riemannian NewTon (NT) and Riemannian Conjugate Gradient descent (CG) are adopted to improve the approximation over a fix rank manifold. We prove that our Hybrid method of ADMM and NT (HADMNT) converges to an optimum of NNLS at least quadratically. The experiments on large-scale collaborative filtering datasets demonstrate very competitive performance of these fast hybrid methods compared to the state-of-the-arts.

Guarantees of Riemannian Optimization for Low Rank Matrix Recovery

We establish theoretical recovery guarantees of a family of Riemannian optimization algorithms for low rank matrix recovery, which is about recovering an $m\times n$ rank $r$ matrix from $p < mn$ number of linear measurements. The algorithms are first interpreted as iterative hard thresholding algorithms with subspace projections. Based on this connection, we show that provided the restricted isometry constant $R_{3r}$ of the sensing operator is less than $C_\kappa /\sqrt{r}$, the Riemannian gradient descent algorithm and a restarted variant of the Riemannian conjugate gradient algorithm are guaranteed to converge linearly to the underlying rank $r$ matrix if they are initialized by one step hard thresholding. Empirical evaluation shows that the algorithms are able to recover a low rank matrix from nearly the minimum number of measurements necessary.

A Dai–Yuan-type Riemannian conjugate gradient method with the weak Wolfe conditions

This article describes a new Riemannian conjugate gradient method and presents a global convergence analysis. The existing Fletcher-Reeves-type Riemannian conjugate gradient method is guaranteed to be globally convergent if it is implemented with the strong Wolfe conditions. On the other hand, the Dai-Yuan-type Euclidean conjugate gradient method generates globally convergent sequences under the weak Wolfe conditions. This article deals with a generalization of Dai-Yuan's Euclidean algorithm to a Riemannian algorithm that requires only the weak Wolfe conditions. The global convergence property of the proposed method is proved by means of the scaled vector transport associated with the differentiated retraction. The results of numerical experiments demonstrate the effectiveness of the proposed algorithm.

Low-rank matrix completion via preconditioned optimization on the Grassmann manifold

We address the numerical problem of recovering large matrices of low rank when most of the entries are unknown. We exploit the geometry of the low-rank constraint to recast the problem as an unconstrained optimization problem on a single Grassmann manifold. We then apply second-order Riemannian trust-region methods (RTRMC 2) and Riemannian conjugate gradient methods (RCGMC) to solve it. A preconditioner for the Hessian is introduced that helps control the conditioning of the problem and we detail preconditioned versions of Riemannian optimization algorithms. The cost of each iteration is linear in the number of known entries. The proposed methods are competitive with state-of-the-art algorithms on a wide range of problem instances. In particular, they perform well on rectangular matrices. We further note that second-order and preconditioned methods are well suited to solve badly conditioned matrix completion tasks.

Approximate Riemannian Conjugate Gradient Learning for Fixed-Form Variational Bayes

Variational Bayesian (VB) methods are typically only applied to models in the conjugate-exponential family using the variational Bayesian expectation maximisation (VB EM) algorithm or one of its va...

Convergence Conditions for Ascent Methods

Convergence Conditions for Ascent Methods