Projected Gradient Approach Research Articles

Constructing low-rank Tucker tensor approximations using generalized completion

Abstract The projected gradient method for matrix completion is generalized towards the higher-dimensional case of low-rank Tucker tensors. It is shown that an operation order rearrangement in the common projected gradient approach provides a complexity improvement. An even better algorithm complexity can be obtained by replacing the completion operator by a general operator that satisfies restricted isometry property; however, such a replacement transforms the completion algorithm into an approximation algorithm.

Energy Efficiency Maximization in Large-Scale Cell-Free Massive MIMO: A Projected Gradient Approach

This paper considers the fundamental power allocation problem in cell-free massive mutiple-input and multiple-output (MIMO) systems which aims at maximizing the total energy efficiency (EE) under a sum power constraint at each access point (AP) and a quality-of-service (QoS) constraint at each user. Existing solutions for this optimization problem are based on solving a sequence of second-order cone programs (SOCPs), whose computational complexity scales dramatically with the network size. Therefore, they are not implementable for practical large-scale cell-free massive MIMO systems. To tackle this issue, we propose an iterative power control algorithm based on the frame work of an accelerated projected gradient (APG) method. In particular, each iteration of the proposed method is done by simple closed-form expressions, where a penalty method is applied to bring constraints into the objective in the form of penalty functions. Finally, the convergence of the proposed algorithm is analytically proved and numerically compared to the known solution based on SOCP. Simulations results demonstrate that our proposed power control algorithm can achieve the same EE as the existing SOCPs-based method, but more importantly, its run time is much lower (one to two orders of magnitude reduction in run time, compared to the SOCPs-based approaches).

Fast multidimensional dictionary learning algorithms and their application in 3D inverse synthetic aperture radar image restoration and noise reduction

By generalising dictionary learning (DL) algorithms to multidimensional (MD) mode and using them in applications where signals are inherently multidimensional, such as in three-dimensional (3D) inverse synthetic aperture radar (ISAR) imaging, it is possible to achieve much higher speed and less computational complexity. In this study, the formulation of the multidimensional dictionary learning (MDDL) problem is expressed and two algorithms are proposed to solve it. The first one is based on the method of optimum directions (MOD) algorithm for 1D dictionary learning (1DDL), which uses alternating minimisation and gradient projection approach. As the MDDL problem is non-convex, the second algorithm approximates the non-convex objective with a new jointly convex function and efficiently solves it. As an application, we use the proposed methods to restore and denoise the ISAR image. Numerical experiments highlight that the proposed algorithms, in addition to reducing the computational complexity and the amount of required memory, also entail less training data for learning the dictionary, and enjoy higher convergence speed in comparison to their one-dimensional (1D) counterparts. Specifically, convergence speed of MD algorithms, depending on the size of the training data, is up to at least 10.7 times faster than the equivalent 1DDL algorithm. According to the simulation results, the SNR value achieved by the proposed algorithms is higher than the case where we use the 3D-IFFT for image reconstruction and the case of fixed dictionaries, by approximately 12 and 4 dB, respectively.

Scaled Consensus and Reference Tracking in Multiagent Networks With Constraints

This paper considers the scaled consensus problem for networks of groups of agents having state constraints. We first address the problem by using a gradient projection approach and design distributed consensus protocols that implement state restriction and steer agents asymptotically to proportions in terms of pre-assigned scales. We then extend the method to multiagent systems with static reference values. We propose and analyze sufficient conditions on agent dynamics such that scaled reference tracking problem is solved. The results are applied to a ship steering system and numerical examples to verify the effectiveness of the proposed cooperative coordination system scheme.

Detecting genetic associations with brain imaging phenotypes in Alzheimer's disease via a novel structured KCCA approach.

Neuroimaging genetics has become an important research topic since it can reveal complex associations between genetic variants (i.e. single nucleotide polymorphisms (SNPs) and the structures or functions of the human brain. However, existing kernel mapping is difficult to directly use the sparse representation method in the kernel feature space, which makes it difficult for most existing sparse canonical correlation analysis (SCCA) methods to be directly promoted in the kernel feature space. To bridge this gap, we adopt a novel alternating projected gradient approach, gradient KCCA (gradKCCA) model to develop a powerful model for exploring the intrinsic associations among genetic markers, imaging quantitative traits (QTs) of interest. Specifically, this model solves kernel canonical correlation (KCCA) with an additional constraint that projection directions have pre-images in the original data space, a sparsity-inducing variant of the model is achieved through controlling the [Formula: see text]-norm of the preimages of the projection directions. We evaluate this model using Alzheimer's disease Neuroimaging Initiative (ADNI) cohort to discover the relationships among SNPs from Alzheimer's disease (AD) risk gene APOE, imaging QTs extracted from structural magnetic resonance imaging (MRI) scans. Our results show that the algorithm not only outperforms the traditional KCCA method in terms of Root Mean Square Error (RMSE) and Correlation Coefficient (CC) but also identify the meaningful and relevant biomarkers of SNPs (e.g. rs157594 and rs405697), which are positively related to right Postcentral and right SupraMarginal brain regions in this study. Empirical results indicate its promising capability in revealing biologically meaningful neuroimaging genetics associations and improving the disease-related mechanistic understanding of AD.

Multi-material topology optimization of compliant mechanisms using regularized projected gradient approach

In this article, the identification of optimal topologies for multi-material compliant mechanisms based on the regularized projected gradient approach is presented and analyzed. The monolithic structure of compliant mechanisms makes them suitable to be used in microelectromechanical systems to transfer load and force. The use of additive manufacturing technologies makes it possible to build multi-material structures. However, a proper design optimization approach must be considered to design multi-material mechanisms as well. In this article, the solid isotropic material with a penalization (SIMP) material interpolation scheme is used to parameterize the continuum design domain. The perimeter penalization approach is utilized to remove the checkerboard patterns and scatters in optimal designs. To handle the volume equality constraint, the regularized constraint projection strategy is utilized. Some benchmark problems are considered to analyze the performance of the method.

Sign projected gradient flow: A continuous-time approach to convex optimization with linear equality constraints

In this paper, we exploit the possibility of combining nonsmooth control jointly with projected gradient approaches to solve convex optimization problems with linear equality constraints. Our development gives rise to a sign projected gradient method employing only “coarse” information, which has finite-time convergence capability and preserves the simplicity and elegance of gradient descent. Our primary objective is to understand the design principle of the projection matrix in the nonsmooth setting and to investigate some key features of the method, such as convergence time. Specifically, we offer a set of convergence results via nonsmooth analysis, revealing that the algorithm guarantees finite-time convergence for a wide range of cost functions, including strongly convex, strictly convex, and convex functions, provided that the proposed design criteria and conditions are satisfied. The results suggest that the choice of the projection matrix might not be unique, which motivates the optimal design issue of the projection matrix to minimize convergence time. We show that the problem can be formulated as a convex optimization problem, which can be solved readily by existing optimization tools. We discuss the possibility of employing the proposed method to solve unconstrained optimization problems, which in turn gives an initial feasible solution to the sign projected gradient flow. We apply the algorithm to solve a formation control problem, and the numerical results show the effectiveness of the algorithm.

Three-Term of New Conjugate Gradient Projection Approach under Wolfe Condition to Solve Unconstrained Optimization Problems

Three-Term of New Conjugate Gradient Projection Approach under Wolfe Condition to Solve Unconstrained Optimization Problems

On proportional fairness of uplink spectral efficiency in cell‐free massive MIMO systems

SummaryThis paper is concerned with the proportional fairness (PF) of the spectral efficiency (SE) maximization of uplinks in a cell‐free (CF) massive multiple‐input multiple‐output (MIMO) system in which a large number of single‐antenna access points (APs) connected to a central processing unit (CPU) serve many single‐antenna users. To detect the user signals, the APs use matched filters based on the local channel state information while the CPU deploys receiver filters based on knowledge of channel statistics. We devise the maximization problem of the SE PF, which maximizes the sum of the logarithm of the achievable user rates, as a jointly nonconvex optimization problem of receiver filter coefficients and user power allocation subject to user power constraints. To handle the challenges associated with the nonconvexity of the formulated design problem, we develop an iterative algorithm by alternatively finding optimal filter coefficients at the CPU and transmit powers at the users. While the filter coefficient design is formulated as a generalized eigenvalue problem, the power allocation problem is addressed by a gradient projection (GP) approach. Simulation results show that the SE PF maximization not only offers approximately the achievable sum rates as compared to the sum‐rate maximization but also provides an improved trade‐off between the user rate fairness and the achievable sum rate.

Similarity learning for person reidentification using projected gradient method

The problem of learning a similarity measure for cross-view person identification in a disjointed camera network is addressed. Our learning framework is based on the projected gradient approach and is suitable for large-scale applications. The hinge loss with the triplet constraints is used as the objective function. Contrary to other approaches where a dual formulation is used, we propose to optimize the objective function in the primal form, with the help of a minibatch gradient descent algorithm. The choice of learning rate and its schedule are nontrivial tasks in such optimization schemes. We have studied empirically the effect of using different strategies for learning rate schedules. Our experimental study includes conventional strategies such as fixed or diminishing learning rates and recently developed adaptive gradient methods such as root-mean-square propagation (RMSProp) for learning rate schedule. The experimental results are presented on three benchmark datasets of person reidentification, namely VIPeR, CUHK Campus, and CUHK03. Our experimental results demonstrate that the RMSProp is well suited for the proposed learning framework. A comparison with recent methods in the domain presents the justification of the proposed approach.

Denoising and deblurring gold immunochromatographic strip images via gradient projection algorithms

Gold immunochromatographic strip (GICS) assay provides a quick, convenient, single-copy and on-site approach to determine the presence or absence of the target analyte when applied to an extensive variety of point-of-care tests. It is always desirable to quantitatively detect the concentration of trace substance in the specimen so as to uncover more useful information compared with the traditional qualitative (or semi-quantitative) strip assay. For this purpose, this paper is concerned with the GICS image denoising and deblurring problems caused by the complicated environment of the intestine/intrinsic restrictions of the strip characteristics and the equipment in terms of image acquisition and transmission. The gradient projection approach is used, together with the total variation minimization approach, to denoise and deblur the GICS images. Experimental results and quantitative evaluation are presented, by means of the peak signal-to-noise ratio, to demonstrate the performance of the combined algorithm. The experimental results show that the gradient projection method provides robust performance for denoising and deblurring the GICS images, and therefore serves as an effective image processing methodology capable of providing more accurate information for the interpretation of the GICS images.

Joint source and relay design for MIMO multi-relay systems using projected gradient approach

Abstract In this paper, we develop the optimal source precoding matrix and relay amplifying matrices for non-regenerative multiple-input multiple-output (MIMO) relay communication systems with parallel relay nodes using the projected gradient (PG) approach. We show that the optimal relay amplifying matrices have a beamforming structure. Exploiting the structure of relay matrices, an iterative joint source and relay matrices optimization algorithm is developed to minimize the mean-squared error (MSE) of the signal waveform estimation at the destination using the PG approach. The performance of the proposed algorithm is demonstrated through numerical simulations.

On the Beamforming Capacity of MISO Channels

This letter addresses the calculation of the ergodic beamforming capacity of a multiple-input single-output (MISO) communication channel in the presence of additive Gaussian noise and known channel distribution information at the transmitter (CDIT). Necessary conditions to achieve the ergodic beamforming capacity are presented in the general case and derived explicitly when the CDIT is Gaussian. This case is commonly addressed in the literature as the joint channel mean information (CMI) and channel covariance information (CCI) case. An iterative algorithm based on the projected gradient approach is derived in order to find the beamforming capacity and the corresponding capacity-achieving beamforming vector.

Simple and Certifiable Quadratic Programming Algorithms for Embedded Linear Model Predictive Control

In this paper we review a dual fast gradient-projection approach to solving quadratic programming (QP) problems recently proposed in [Patrinos and Bemporad, 2012] that is particularly useful for embedded model predictive control (MPC) of linear systems subject to linear constraints on inputs and states. We show that the method has a computational effort aligned with several other existing QP solvers typically used in MPC, and in addition it is extremely easy to code, requires only basic and easily parallelizable arithmetic operations, and a number of iterations to reach a given accuracy in terms of optimality and feasibility of the primal solution that can be estimated quite tightly by solving an off-line mixed-integer linear programming problem.

DINDSCAL: direct INDSCAL

The well-known INDSCAL model for simultaneous metric multidimensional scaling (MDS) of three-way data analyzes doubly centered matrices of squared dissimilarities. An alternative approach, called for short DINDSCAL, is proposed for analyzing directly the input matrices of squared dissimilarities. An important consequence is that missing values can be easily handled. The DINDSCAL problem is solved by means of the projected gradient approach. First, the problem is transformed into a gradient dynamical system on a product matrix manifold (of Stiefel sub-manifold of zero-sum matrices and non-negative diagonal matrices). The constructed dynamical system can be numerically integrated which gives a globally convergent algorithm for solving the DINDSCAL. The DINDSCAL problem and its solution are illustrated by well-known data routinely used in metric MDS and INDSCAL. Alternatively, the problem can also be solved by iterative algorithm based on the conjugate (projected) gradient method, which MATLAB implementation is enclosed as an appendix.

Calculation of the Critical Energy of the Power System with Protection Operation

Whether a disturbance will trigger unexpected protection operation to exacerbate the normal operation state or even lead to the unstable state depends on the transient energy that the system gains during the fault period. The paper presents a new gradient projection approach to calculate the critical energy for transient stability assessment of the power system with the protection operation. The method takes the projection of the potential energy gradient on the key protection operation surface as search direction and iteratively searches for the critical energy at the surface; in order to let the optimal point be the feasible point, the generator angles are considered as control variables of power flow equation to solve network bus voltage and angle during the searching process. Examples show that the convergence and precision of the new method proposed is much better than that of other methods, which demonstrates the validity and feasibility of the new method.

Fast Nonnegative Matrix Factorization Algorithms Using Projected Gradient Approaches for Large-Scale Problems

Recently, a considerable growth of interest in projected gradient (PG) methods has been observed due to their high efficiency in solving large-scale convex minimization problems subject to linear constraints. Since the minimization problems underlying nonnegative matrix factorization (NMF) of large matrices well matches this class of minimization problems, we investigate and test some recent PG methods in the context of their applicability to NMF. In particular, the paper focuses on the following modified methods: projected Landweber, Barzilai-Borwein gradient projection, projected sequential subspace optimization (PSESOP), interior-point Newton (IPN), and sequential coordinate-wise. The proposed and implemented NMF PG algorithms are compared with respect to their performance in terms of signal-to-interference ratio (SIR) and elapsed time, using a simple benchmark of mixed partially dependent nonnegative signals.

Advances in Nonnegative Matrix and Tensor Factorization

Advances in Nonnegative Matrix and Tensor Factorization

MULTILAYER NONNEGATIVE MATRIX FACTORIZATION USING PROJECTED GRADIENT APPROACHES

The most popular algorithms for Nonnegative Matrix Factorization (NMF) belong to a class of multiplicative Lee-Seung algorithms which have usually relative low complexity but are characterized by slow-convergence and the risk of getting stuck to in local minima. In this paper, we present and compare the performance of additive algorithms based on three different variations of a projected gradient approach. Additionally, we discuss a novel multilayer approach to NMF algorithms combined with multi-start initializations procedure, which in general, considerably improves the performance of all the NMF algorithms. We demonstrate that this approach (the multilayer system with projected gradient algorithms) can usually give much better performance than standard multiplicative algorithms, especially, if data are ill-conditioned, badly-scaled, and/or a number of observations is only slightly greater than a number of nonnegative hidden components. Our new implementations of NMF are demonstrated with the simulations performed for Blind Source Separation (BSS) data.

Reserve capacity of signal-controlled road network

For a signal control road network subject to equilibrium flows, the maximum possible increase in travel demands is considered in this paper. Using the concept of reserve capacity of signal-controlled junctions, the problem of finding the maximum increase in traffic demands can be formulated as a mathematical program with equilibrium constraints (MPEC). In this paper, we present a projected gradient approach to obtain the maximum increase in travel demands based on the TRANSYT traffic model. Numerical computations are made on a grid network where good results are obtained.