Gradient Projection Algorithm Research Articles

Acceleration of a projected gradient algorithm for the Bingham flow problem by rigidity enforcement with penalty

Acceleration of a projected gradient algorithm for the Bingham flow problem by rigidity enforcement with penalty

Distributed stochastic optimization algorithm with non-consistent constraints in time-varying unbalanced networks

This study focuses on a class of distributed optimization problems with non-consistent local set constraints in time-varying unbalanced networks. Considering that each agent only has access to local gradient information with random errors, this paper proposes a distributed stochastic gradient projection algorithm. Under some conditions of random errors and a uniformly joint strongly connected topology, it is shown that the local decision states of all agents converge to a common optimal solution with a probability of one, achieving a sublinear convergence rate of O(lnk/k). Notably, under the condition that at least one local function exhibits strong convexity, the algorithm achieves a faster sublinear convergence rate of O(1/k). These theoretical results are validated through numerical simulations. Furthermore, this paper applies the proposed algorithm to solve for the unknown parameters in distributed linear regression problems with incomplete data. Numerical results indicate that the algorithm not only aligns with the convergence properties reported in existing literature but also demonstrates significant superiority in convergence speed. This important discovery not only lays a solid theoretical foundation for the further application and extension of the algorithm but also reveals its immense potential in solving practical problems.

Prescribed-time distributed optimization problem with constraints

In recent years, distributed optimization problem have a wide range of applications in various fields. This paper considers the prescribed-time distributed optimization problem with/without constraints. Firstly, we assume the state of each agent is constrained, and the prescribed-time distributed optimization algorithm with constraints is designed on the basis of gradient projection algorithm and consensus algorithm. Secondly, the constrained distributed optimization problem is transformed into the unconstrained distributed optimization problem, and according to the gradient descent algorithm and consensus algorithm, we also propose the prescribed-time distributed optimization algorithm without constraints. By designing the appropriate objective functions, we prove the multi-agent system can converge to the optimal solution within any prescribed-time, and the convergence time is fully independent of the initial conditions and system parameters. Finally, three simulation examples are provided to verify the validity of the designed algorithms.

An accelerated relaxed-inertial strategy based CGP algorithm with restart technique for constrained nonlinear pseudo-monotone equations to image de-blurring problems

In this paper, an accelerated spectral conjugate gradient projection algorithm is proposed for solving the constrained nonlinear pseudo-monotone equations. We set a restart procedure related to the conjugate parameter in the search direction of the spectral conjugate gradient method. We use an accelerated gradient descent scheme to generate the spectral parameter. Finally, we adopt the relaxed-inertial strategy to accelerated algorithm iteration process. Global convergence is proved under mild conditions without the Lipschitz continuity assumption. Numerical comparisons with other methods show the performances of our algorithm for solving large-scale equations. Our applications are on regularized decentralized logistic regression and image de-blurring problems.

Alternating projection combined with fast gradient projection (FGP-AP) method for intensity-only measurement optical diffraction tomography in LED array microscopy.

Optical diffraction tomography (ODT) is a powerful label-free measurement tool that can quantitatively image the three-dimensional (3D) refractive index (RI) distribution of samples. However, the inherent "missing cone problem," limited illumination angles, and dependence on intensity-only measurements in a simplified imaging setup can all lead to insufficient information mapping in the Fourier domain, affecting 3D reconstruction results. In this paper, we propose the alternating projection combined with the fast gradient projection (FGP-AP) method to compensate for the above problem, which effectively reconstructs the 3D RI distribution of samples using intensity-only images captured from LED array microscopy. The FGP-AP method employs the alternating projection (AP) algorithm for gradient descent and the fast gradient projection (FGP) algorithm for regularization constraints. This approach is equivalent to incorporating prior knowledge of sample non-negativity and smoothness into the 3D reconstruction process. Simulations demonstrate that the FGP-AP method improves reconstruction quality compared to the original AP method, particularly in the presence of noise. Experimental results, obtained from mouse kidney cells and label-free blood cells, further affirm the superior 3D imaging efficacy of the FGP-AP method.

Closest Farthest Widest

The current paper proposes and tests algorithms for finding the diameter of a compact convex set and the farthest point in the set to another point. For these two nonconvex problems, I construct Frank–Wolfe and projected gradient ascent algorithms. Although these algorithms are guaranteed to go uphill, they can become trapped by local maxima. To avoid this defect, I investigate a homotopy method that gradually deforms a ball into the target set. Motivated by the Frank–Wolfe algorithm, I also find the support function of the intersection of a convex cone and a ball centered at the origin and elaborate a known bisection algorithm for calculating the support function of a convex sublevel set. The Frank–Wolfe and projected gradient algorithms are tested on five compact convex sets: (a) the box whose coordinates range between −1 and 1, (b) the intersection of the unit ball and the non-negative orthant, (c) the probability simplex, (d) the Manhattan-norm unit ball, and (e) a sublevel set of the elastic net penalty. Frank–Wolfe and projected gradient ascent are about equally fast on these test problems. Ignoring homotopy, the Frank–Wolfe algorithm is more reliable. However, homotopy allows projected gradient ascent to recover from its failures.

Assignment of Freight Traffic in a Large-scale Intermodal Network under Uncertainty

This paper presents a methodology for freight traffic assignment in a large-scale road-rail intermodal network under uncertainty. Network uncertainties caused by natural disasters have dramatically increased in recent years. Several of these disasters (e.g., Hurricane Sandy, Mississippi River Flooding, and Hurricane Harvey) severely disrupted the U.S. freight transportation network, and consequently, the supply chain. To account for these network uncertainties, a stochastic freight traffic assignment model is formulated. An algorithmic framework, involving the sample average approximation and gradient projection algorithm, is proposed to solve this challenging problem. The developed methodology is tested on the U.S. intermodal network with freight flow data from the Freight Analysis Framework. The experiments consider three types of natural disasters that have different risks and impacts on transportation networks: earthquakes, hurricanes, and floods. It is found that for all disaster scenarios, freight ton-miles are higher compared to the base case without uncertainty. The increase in freight ton-miles is the highest under the flooding scenario; this is because there are more states in the flood-risk areas, and they are scattered throughout the U.S.

Range-constrained traffic assignment for electric vehicles under heterogeneous range anxiety

This paper studied the range-constrained traffic assignment problem (RTAP), where heterogeneous range anxiety is considered among the driving population by electric vehicles (EVs). In order not to get stranded en-route, each EV driver is assumed to have his/her own driving range limit for being able to complete the trip. As a result, two types of multi-class RTAP can be defined through discrete or continuous distributed range anxiety. Given path-based side constraint structures, we proposed two variational inequality (VI) formulations for modeling discrete and continuous RTAPs, where the former is finite-dimensional according to a discrete number of user classes and the latter is infinite-dimensional accounting for an infinite number of user classes. We reformulate the continuous RTAP into finite-dimensional by merging adjacent EV drivers into one group. A unified path-based solution framework is developed to solve the two RTAPs, built upon the gradient projection algorithm. We design column generation and dropping schemes to adaptively maintain compact path sets and an inner equilibration strategy to accelerate convergence. Finally, a small network is used to examine the correctness and effectiveness of proposed models, and a large Winnipeg network is adopted to evaluate the impacts of stochastic driving range on network flows and computation costs. Numerical results provide compelling evidence of the outstanding superiority of the proposed algorithm, and show that EV drivers with heightened sensitivity towards range anxiety may contribute to the emergence of critical links experiencing severe congestion.

Computation of the Hausdorff Distance between Two Compact Convex Sets.

The Hausdorff distance between two closed sets has important theoretical and practical applications. Yet apart from finite point clouds, there appear to be no generic algorithms for computing this quantity. Because many infinite sets are defined by algebraic equalities and inequalities, this a huge gap. The current paper constructs Frank-Wolfe and projected gradient ascent algorithms for computing the Hausdorff distance between two compact convex sets. Although these algorithms are guaranteed to go uphill, they can become trapped by local maxima. To avoid this defect, we investigate a homotopy method that gradually deforms two balls into the two target sets. The Frank-Wolfe and projected gradient algorithms are tested on two pairs of compact convex sets, where: (1) is the box translated by and is the intersection of the unit ball and the non-negative orthant; and (2) is the probability simplex and is the unit ball translated by . For problem (2), we find the Hausdorff distance analytically. Projected gradient ascent is more reliable than the Frank-Wolfe algorithm and finds the exact solution of problem (2). Homotopy improves the performance of both algorithms when the exact solution is unknown or unattained.

A new nonmonotone spectral projected gradient algorithm for box-constrained optimization problems in [formula omitted] real matrix space with application in image clustering

Box-constrained optimization problems in the real m×n matrix space have been widely applied in big data mining. However, efficient solution of them is still a challenge. In this paper, a new nonmonotone line search rule is first proposed by extending the well-known ones and inheriting their advantages. Then, by analyzing and exploiting properties of this rule, a new nonmonotone spectral projected gradient algorithm is developed to solve the box-constrained optimization problems in the matrix space. Global convergence of the developed algorithm is also established. Numerical tests are conducted on a series of randomly generated test problems and those in the set of benchmark test problems. Compared with other existing nonmonotone line search rules, our rule shows its advantages in terms of the significantly reduced number of function evaluations and significantly reduced number of iterations. To further validate applicability of this research, we apply the studied optimization problem and the developed algorithm to solve the problems of image clustering. Numerical results demonstrate that the proposed method can generate better clustering results and is more robust than the similar ones available in the literature.

A convergence analysis of hybrid gradient projection algorithm for constrained nonlinear equations with applications in compressed sensing

A convergence analysis of hybrid gradient projection algorithm for constrained nonlinear equations with applications in compressed sensing

Algorithm 1035: A Gradient-based Implementation of the Polyhedral Active Set Algorithm

The Polyhedral Active Set Algorithm (PASA) is designed to optimize a general nonlinear function over a polyhedron. Phase one of the algorithm is a nonmonotone gradient projection algorithm, while phase two is an active set algorithm that explores faces of the constraint polyhedron. A gradient-based implementation is presented, where a projected version of the conjugate gradient algorithm is employed in phase two. Asymptotically, only phase two is performed. Comparisons are given with IPOPT using polyhedral-constrained problems from CUTEst and the Maros/Meszaros quadratic programming test set.

A novel flow update policy in solving traffic assignment problems: Successive over relaxation iteration method

This paper presents a novel flow update policy, namely the successive over relaxation (SOR) iteration method, which can be implemented in traffic assignment algorithms. Most existing solution algorithms for the user equilibrium traffic assignment problem (UE-TAP) mainly use two flow update policies: Jacobi and Gauss-Seidel iteration methods. The proposed flow update policy SOR can be a more efficient replacement. Following the path-based gradient projection (GP) algorithm, we developed a new method GP-SOR for the UE-TAP. This study first provides the complete procedure of applying the GP-SOR algorithm to solve the UE-TAP. Subsequently, a few properties of the proposed method are rigorously proven. However, empirical tests of the GP-SOR algorithm demonstrate serious oscillations and poor convergence. To cope with this problem, the Armijo Rule is employed to determine the relaxation factor, which substantially improves the convergence of GP-SOR algorithm. The preliminary numerical examples show that the GP-SOR algorithm has speedier convergence compared with the known alternatives, which is reflected by the evident reduction of the computing time and the number of iterations.

An alternating direction method of multipliers for solving user equilibrium problem

This paper introduces a new parallel computing algorithm to address the user equilibrium (UE) problem. Searching for efficient solution algorithms for UE has been a recurring study subject in transportation research and has attracted much attention in past decades. Existing solution algorithms can be classified into three categories: link-based, path-based, and origin-based. This paper introduces an alternating direction method of multipliers (ADMM) algorithm that is different from these categories. Based on the origin-based formulation of UE problem, an equivalent problem is proposed which eliminates the flow conservation conditions through the augmented Lagrangian function. In order to make use of the ADMM, the network links should be grouped into different blocks, where the links in the same block are disconnected. This link grouping problem falls into the category of edge-coloring problem in graph theory, and it follows the Vizing theorem. A novel approach is developed for the link grouping problem. For links in the same block, we have a separable subproblem, which is solved in parallel by the gradient projection algorithm. Numerical experiments are conducted to validate the proposed algorithm, which shows its computation efficiency.

Research on trajectory planning of mobile manipulator based on improved gradient projection algorithm

This paper takes a 10 DOF mobile manipulator as the research object. Aiming at the problem of joint acceleration overrun in the trajectory planning of a 10 DOF Manipulator, firstly, the scale factor is obtained by the extremum method. Compared with the commonly used extremum method, the optimization degree and the self motion limit factor are introduced, and a new algorithm considering joint acceleration constraint is deduced and proposed. By comparing the simulation results of the inverse solution algorithm, the effectiveness of the new algorithm is verified. Finally, it is applied to a mobile manipulator with 10 degrees of freedom for example motion test. The test results meet the design requirements. The results will be actually applied in engineering projects.

Rotation to Sparse Loadings Using L^p Losses and Related Inference Problems

Researchers have widely used exploratory factor analysis (EFA) to learn the latent structure underlying multivariate data. Rotation and regularised estimation are two classes of methods in EFA that they often use to find interpretable loading matrices. In this paper, we propose a new family of oblique rotations based on component-wise L^p loss functions (0 < ple 1) that is closely related to an L^p regularised estimator. We develop model selection and post-selection inference procedures based on the proposed rotation method. When the true loading matrix is sparse, the proposed method tends to outperform traditional rotation and regularised estimation methods in terms of statistical accuracy and computational cost. Since the proposed loss functions are nonsmooth, we develop an iteratively reweighted gradient projection algorithm for solving the optimisation problem. We also develop theoretical results that establish the statistical consistency of the estimation, model selection, and post-selection inference. We evaluate the proposed method and compare it with regularised estimation and traditional rotation methods via simulation studies. We further illustrate it using an application to the Big Five personality assessment.

Неравенство Лежанского – Поляка – Лоясевича и сходимость метода проекции градиента

Неравенство Лежанского – Поляка – Лоясевича и сходимость метода проекции градиента

A Multilevel Monte Carlo Approach for a Stochastic Optimal Control Problem Based on the Gradient Projection Method

A multilevel Monte Carlo (MLMC) method is applied to simulate a stochastic optimal problem based on the gradient projection method. In the numerical simulation of the stochastic optimal control problem, the approximation of expected value is involved, and the MLMC method is used to address it. The computational cost of the MLMC method and the convergence analysis of the MLMC gradient projection algorithm are presented. Two numerical examples are carried out to verify the effectiveness of our method.

Globally Optimized TDOA High-Frequency Source Localization Based on Quasi-Parabolic Ionosphere Modeling and Collaborative Gradient Projection

We investigate the problem of high-frequency (HF) source localization using the time-difference-of-arrival (TDOA) observations of ionosphere-refracted radio rays based on quasi-parabolic (QP) modeling. An unresolved but pertinent issue in such a field is that the existing gradient-type scheme can easily get trapped in local optima for practical use. This will lead to the difficulty in initializing the algorithm and finally degraded positioning performance if the starting point is inappropriately selected. In this article, we develop a collaborative gradient projection (GP) algorithm in order to globally solve the highly nonconvex QP-based TDOA HF localization problem. The metaheuristic of particle swarm optimization (PSO) is exploited for information sharing among multiple GP models, each of which is guaranteed to work out a critical point solution to the simplified maximum likelihood formulation. Random mutations are incorporated to avoid the early convergence of PSO. Rather than treating the geolocation of HF transmitter as a pure optimization problem, we further provide workarounds for addressing the possible impairments and challenges when the proposed technique is applied in practice. Numerical results demonstrate the effectiveness of our PSO-assisted reinitialization strategy in achieving the global optimality, and the superiority of our method over its competitor in terms of positioning accuracy.

Spectral approximation for optimal control problems governed by first bi‐harmonic equation

AbstractWe investigate in the article the L2‐norm state constrained control problem with first bi‐harmonic equation. First, the optimality conditions of the control problem are derived carefully, and spectral discretization of the problem is established. Based on the property of projection operator, we establish a priori error analysis for control variable, state, and adjoint state variable. Furthermore, the efficient projected gradient algorithm is proposed, and the numerical results we obtained verify the analytical results that it can provide high order accuracy and fast convergence rate.