Riemannian Conjugate Gradient Method Research Articles

On the convergence of orthogonalization-free conjugate gradient method for extreme eigenvalues of Hermitian matrices: A Riemannian optimization interpretation

In many applications, it is desired to obtain extreme eigenvalues and eigenvectors of large Hermitian matrices by efficient and compact algorithms. In particular, orthogonalization-free methods are preferred for large-scale problems for finding eigenspaces of extreme eigenvalues without explicitly computing orthogonal vectors in each iteration. For the top p eigenvalues, the simplest orthogonalization-free method is to find the best rank-p approximation to a positive semi-definite Hermitian matrix by algorithms solving the unconstrained Burer–Monteiro formulation. We show that the nonlinear conjugate gradient method for the unconstrained Burer–Monteiro formulation is equivalent to a Riemannian conjugate gradient method on a quotient manifold with the Bures–Wasserstein metric, thus its global convergence to a stationary point can be proven. Numerical tests suggest that it is efficient for computing the largest k eigenvalues for large-scale matrices if the largest k eigenvalues are nearly distributed uniformly.

An accelerated hybrid Riemannian conjugate gradient method for unconstrained optimization

With the goal to deal with a series of optimization problems on general matrix manifolds with differentiable objective functions, we propose an accelerated hybrid Riemannian conjugate gradient technique. Specifically, the acceleration scheme of the proposed method using a modified stepsize which is multiplicatively determined by the Wolfe line search. The search direction of the proposed algorithm is determined by the hybrid conjugate parameter with computationally promising. We showed that the suggested approach converges globally to a stationary point. Our approach performs better than the state of art Riemannian conjugate gradient algorithms, as illustrated by computations on problems such as the orthogonal Procrustes problem and the Brockett-cost-function minimization problem.

Riemannian Conjugate Gradient Method Based on Exact Penalty for Phase-Only Beamforming Synthesis

Riemannian Conjugate Gradient Method Based on Exact Penalty for Phase-Only Beamforming Synthesis

Hybrid/Reflective Beamforming for IRS-Assisted Dual-Function Radar-Communication System

This paper investigates the joint design problem of the hybrid/reflective beamforming for the dual-function radar-communication (DRC) system with the help of an intelligent reconfigurable surface (IRS). The degrees of freedom for beamformer design are the analog and digital beamformers at the base station (BS), and the reflective beamformer at the IRS. To begin with, we formulate the beamformer design problem as a weighted sum-rate (WSR) maximization problem subject to the transmit power and similarity between the designed beamformer and the reference one with desired beampattern property. Then, utilizing equivalent reformulation of the preceding design problem, we conceive an alternating direction sequential programming (ADSP) based algorithm that iteratively updates the following low-complexity subproblems: 1) digital beamformer at the BS through the Powell-Hestenes-Rockafellar (PHR) method; 2) analog beamformer at the BS using the riemannian conjugate gradient method; and 3) reflective beamformer at the IRS by the successive convex approximation (SCA) technique. The convergence is ensured by analytically offering the non-decreasing and boundary of the WSR value. Finally, numerical results verify the effectiveness of the proposed algorithm, which achieves superior WSR performance while ensuring the radar requirement.

An improved Riemannian conjugate gradient method and its application to robust matrix completion

An improved Riemannian conjugate gradient method and its application to robust matrix completion

Riemannian conjugate gradient method for low-rank tensor completion

Riemannian conjugate gradient method for low-rank tensor completion

Global convergence of Hager–Zhang type Riemannian conjugate gradient method

This paper presents the Hager–Zhang (HZ)-type Riemannian conjugate gradient method that uses the exponential retraction. We also present global convergence analyses of our proposed method under two kinds of assumptions. Moreover, we numerically compare our proposed methods with the existing methods by solving two kinds of Riemannian optimization problems on the unit sphere. The numerical results show that our proposed method has much better performance than the existing methods, i.e., the FR, DY, PRP, and HS methods. In particular, they show that it has much higher performance than existing methods including the hybrid ones in computing the stability number of graphs problem.

NOMA Enhanced Hybrid RIS-UAV-Assisted Full-Duplex Communication System With Imperfect SIC and CSI

In this work, we consider a hybrid aerial full-duplex (FD) relaying protocol consisting of a reconfigurable intelligent surface (RIS) mounted over an FD unmanned aerial vehicle (UAV) relay operating in the decode and forward mode to assist the information transfer between the base station and multiple users. For better spectral efficiency, we investigate the use of non-orthogonal multiple access (NOMA) in such networks and focus on both the performance analysis and design optimization of the considered RIS-NOMA network under imperfect channel state information (CSI) and successive interference cancellation (SIC) at each user, and residual-self interference (RSI) at UAV. We first formulate the sum rate maximization problem and adopt the block coordinate descent method to deal with the non-convex nature of the problem. Thereafter, we propose an algorithm based on the Riemannian conjugate gradient method to get the optimal phase shifts at the RIS, an iterative algorithm to obtain the optimal UAV/RIS position and the exhaustive method to obtain the optimum power allocation coefficients. Next, with obtained optimal position, phase shift and power coefficients, we further analyze the performance of the network and derive the closed-form expressions of outage probability, achievable throughput and ergodic capacity. We present Monte Carlo simulation-based results to validate the accuracy of the proposed algorithms and derived expressions and demonstrate the superiority of NOMA over OMA.

Unsupervised Deep-Learning-Based Reconfigurable Intelligent Surface-Aided Broadcasting Communications in Industrial IoTs

This article presents a general system framework that lays the foundation for reconfigurable intelligent surface (RIS)-enhanced broadcast communications in Industrial Internet of Things (IIoTs). In our system model, we consider multiple sensor clusters co-existing in a smart factory where the direct links between these clusters and a central base station (BS) are blocked completely. In this context, a RIS is utilized to reflect signals broadcast from BS toward cluster heads (CHs) which act as a representative of clusters, where BS only has access to the statistical distribution of the channel state information (CSI). An analytical upper bound of the total ergodic spectral efficiency (SE) and an approximation of outage probability are derived. Based on these analytical results, two algorithms are introduced to control the phase shifts at RIS, which are the Riemannian conjugate gradient (RCG) method and the deep neural network (DNN) method. While the RCG algorithm operates based on the conventional iterative method, and the DNN technique relies on unsupervised deep learning (DL). Our numerical results show that both algorithms achieve satisfactory performance based on only statistical CSI. In addition, compared to the RCG scheme, using DL reduces the computational latency by more than ten times with an almost identical total ergodic SE achieved. These numerical results reveal that while using the conventional RCG method may provide unsatisfactory latency, and the DNN technique shows much promise for enabling RIS in ultrareliable and low-latency communications (URLLC) in the context of IIoTs.

Robust Precoding for 3D Massive MIMO Configuration With Matrix Manifold Optimization

This paper investigates robust downlink precoding for three-dimensional (3D) massive multi-input multi-output (MIMO) configuration with matrix manifold optimization. Starting with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a posteriori</i> channel model, we formulate the robust precoder design to maximize an upper bound of ergodic weighted sum-rate under a total power budget. We derive the generalized eigenvector structure for optimal precoder with matrix manifold optimization. However, since the precoding of multiple users is coupled in the structure, we maximize the objective function for each user in alternation and prove the solution of each individual problem is the generalized eigenvector corresponding to the maximum generalized eigenvalue. In accordance with this, we design an iterative algorithm and present its convergence analysis. Furthermore, we propose a Riemannian conjugate gradient (RCG) method to solve the generalized eigenvalue problem (GEP) for higher efficiency in the precoder design algorithm.

HSH-norm optimal MOR for the MIMO linear time-invariant systems on the Stiefel manifold

This paper focuses on the Hilbert–Schmidt-Hankel-norm ( norm) optimal model order reduction (MOR) of large-scale multiple-input multiple-output (MIMO) linear time-invariant (LTI) systems on the Stiefel manifold. First, a cost function is constructed in regard to the norm. By introducing the orthogonality constraints, the norm optimal MOR problem is converted into an unconstrained minimisation problem on the Stiefel manifold. We derive the Riemannian gradient of the cost function on the Stiefel manifold. The MOR algorithm is developed associated with the Riemannian conjugate gradient method. Global convergence is guaranteed with mild conditions. Finally, the effectiveness of the proposed method is illustrated by two numerical examples.

Transportless conjugate gradient for optimization on Stiefel manifold

In this paper, we focus on building an optimization scheme over the Stiefel manifold that maintains each iterate feasible. We focus on conjugate gradient methods and compare our scheme to the Riemannian optimization approach. We parametrize the Stiefel manifold using the polar decomposition to build an optimization problem over a vector space, instead of a Riemannian manifold. The result is a conjugate gradient method that averts the use of a vector transport, needed in the Riemannian conjugate gradient method. The performance of our method is tested on a variety of numerical experiments and compared with those of three Riemannian optimization methods.

Matrix Factorization for Collaborative Budget Allocation

This paper studies the collaborative budget allocation problem in which users are not isolated in the collaborative consumption of goods or services when available goods or services are limited. Different from existing methods that treat each user independently, we investigate the geometric properties of user’s consumption or preference on services, and design a matrix completion framework on the simplex. In this framework, an item’s allocation vector indicating how available services are allocated to users is estimated by the combination of user profiles as basis points on the simplex. Instead of using Euclidean distance directly, we specify a Riemannian distance on the simplex or project histogram data on simplex to Euclidean space. To intensify our model’s stability, we relax the exact recovery constraint to make a robust collaborative prediction. The resulting objective function is then efficiently optimized by a Riemannian conjugate gradient method on the simplex. Experiments on real-world data sets demonstrate our model’s competitiveness versus other collaborative budget prediction methods. Comparisons of different distance metrics for histogram data are shown and discussed. Note to Practitioners —This paper was motivated by the collaborative budget allocation (CBA) problem in which either a user has limited ratings that can be distributed among items, or a service has limited availability to all users in the sharing economy system. Existing approaches rarely consider this phenomenon in their collaborative prediction modeling, which implies the geometric constraints on a user’s rating vector. This paper suggests a matrix factorization framework to address this problem by learning a group of a user profile as basis points that can be combined to recover other users’ rating vectors. Considering the noise in observed ratings in real applications, we enhance the robustness of collaborative prediction by relaxing the recovery constraint. We then propose to specify the distance metric on the simplex from two perspectives, and give the optimization approach. In the future research, we will address the CBA in more complex settings, for example, some items can be substitutes or complementary for each other, in this way, how items can be better distributed among users is an interesting problem.

Riemannian Optimal System Identification Algorithm for Linear MIMO Systems

In this letter, a novel optimization method for the data-driven local coordinates approach to a prediction error method, which is one of system identification methods, is developed. In the proposed method, parameters of system matrices to be estimated are kept as their original forms of matrices whereas they are converted into a vector in the literature. Optimization with respect to the matrix variables can improve the computational efficiency. Furthermore, it is effective to equate input–output equivalent systems with each other since they attain the same value of the objective function. To this end, a Riemannian quotient manifold is introduced. The geometry of the proposed manifold is investigated and a novel Riemannian conjugate gradient method on the manifold is provided. Numerical experiments show that the difference of the proposed Riemannian conjugate gradient method from Euclidean one greatly improves the efficiency of the algorithm.

Computation of Ground States of the Gross--Pitaevskii Functional via Riemannian Optimization

In this paper we combine concepts from Riemannian Optimization and the theory of Sobolev gradients to derive a new conjugate gradient method for direct minimization of the Gross-Pitaevskii energy functional with rotation. The conservation of the number of particles constrains the minimizers to lie on a manifold corresponding to the unit $L^2$ norm. The idea developed here is to transform the original constrained optimization problem to an unconstrained problem on this (spherical) Riemannian manifold, so that fast minimization algorithms can be applied as alternatives to more standard constrained formulations. First, we obtain Sobolev gradients using an equivalent definition of an $H^1$ inner product which takes into account rotation. Then, the Riemannian gradient (RG) steepest descent method is derived based on projected gradients and retraction of an intermediate solution back to the constraint manifold. Finally, we use the concept of the Riemannian vector transport to propose a Riemannian conjugate gradient (RCG) method for this problem. It is derived at the continuous level based on the "optimize-then-discretize" paradigm instead of the usual "discretize-then-optimize" approach, as this ensures robustness of the method when adaptive mesh refinement is performed in computations. We evaluate various design choices inherent in the formulation of the method and conclude with recommendations concerning selection of the best options. Numerical tests demonstrate that the proposed RCG method outperforms the simple gradient descent (RG) method in terms of rate of convergence. While on simple problems a Newton-type method implemented in the {\tt Ipopt} library exhibits a faster convergence than the (RCG) approach, the two methods perform similarly on more complex problems requiring the use of mesh adaptation. At the same time the (RCG) approach has far fewer tunable parameters.

A Riemannian conjugate gradient method for optimization on the Stiefel manifold

In this paper we propose a new Riemannian conjugate gradient method for optimization on the Stiefel manifold. We introduce two novel vector transports associated with the retraction constructed by the Cayley transform. Both of them satisfy the Ring-Wirth nonexpansive condition, which is fundamental for convergence analysis of Riemannian conjugate gradient methods, and one of them is also isometric. It is known that the Ring-Wirth nonexpansive condition does not hold for traditional vector transports as the differentiated retractions of QR and polar decompositions. Practical formulae of the new vector transports for low-rank matrices are obtained. Dai's nonmonotone conjugate gradient method is generalized to the Riemannian case and global convergence of the new algorithm is established under standard assumptions. Numerical results on a variety of low-rank test problems demonstrate the effectiveness of the new method.

A Dai–Yuan-type Riemannian conjugate gradient method with the weak Wolfe conditions

This article describes a new Riemannian conjugate gradient method and presents a global convergence analysis. The existing Fletcher-Reeves-type Riemannian conjugate gradient method is guaranteed to be globally convergent if it is implemented with the strong Wolfe conditions. On the other hand, the Dai-Yuan-type Euclidean conjugate gradient method generates globally convergent sequences under the weak Wolfe conditions. This article deals with a generalization of Dai-Yuan's Euclidean algorithm to a Riemannian algorithm that requires only the weak Wolfe conditions. The global convergence property of the proposed method is proved by means of the scaled vector transport associated with the differentiated retraction. The results of numerical experiments demonstrate the effectiveness of the proposed algorithm.