Equality-constrained Quadratic Programming Research Articles

Memristive Recurrent Neural Network Circuit for Fast Solving Equality-Constrained Quadratic Programming With Parallel Operation

Equality-constrained quadratic programming (QP) has been one of the most basic and typical problems in the Internet of Things domain. In big data scenarios, how to quickly and accurately solve the problem in hardware has not been realized. Therefore, in this article, a memristive recurrent neural circuit that can parallel solve the QP problem in real time is proposed. First, a new memristive synaptic array is designed that can simultaneously implement parallel reading and writing. On the basis of this structure, a new neural network circuit based on memristor is designed that can perform large-scale recursive operations by parallel methods. This circuit can solve the equality-constrained QP problem in different situations by using such real-time programmable memristor arrays processing in memory. The PSpice simulation results show that the problem can be solved with 99.8% precision. Based on practical verification, the neural circuit experiment on PCB is presented with 97.34% precision. Moreover, the circuit has good robustness under the interference of weight value. And, it has an advantage in processing time compared with FPGA.

A proportional-integral iterative algorithm for time-variant equality-constrained quadratic programming problem with applications

A proportional-integral iterative algorithm for time-variant equality-constrained quadratic programming problem with applications

An augmented Lagrangian filter method

We introduce a filter mechanism to enforce convergence for augmented Lagrangian methods for nonlinear programming. In contrast to traditional augmented Lagrangian methods, our approach does not require the use of forcing sequences that drive the first-order error to zero. Instead, we employ a filter to drive the optimality measures to zero. Our algorithm is flexible in the sense that it allows for equality-constrained quadratic programming steps to accelerate local convergence. We also include a feasibility restoration phase that allows fast detection of infeasible problems. We provide a convergence proof that shows that our algorithm converges to first-order stationary points. We provide preliminary numerical results that demonstrate the effectiveness of our proposed method.

Discrete ZNN models of Adams-Bashforth (AB) type solving various future problems with motion control of mobile manipulator

Zhang neural network (ZNN), being a special type of recurrent neural network, has shown powerful abilities to solve a great variety of continuous time-varying problems. In order to solve future problems (or termed discrete time-varying problems), discrete ZNN (DZNN) models should be developed. In this paper, four kinds of future problems, i.e., future linear system (FLS), future division (FD), future quadratic minimization (FQM), and future equality-constrained quadratic programming (FECQP), are investigated. The DZNN models of Adams-Bashforth (AB) type are thus proposed for solving four kinds of future problems. Compared with conventional DZNN models of Euler type and Taylor type, the performances of DZNN models of AB type in terms of accuracy are better. Meanwhile, the numerical results substantiate the efficacy and superiority of DZNN models of AB type for solving future problems. Furthermore, the motion control of mobile robot manipulator is conducted to substantiate the efficacy of DZNN models of AB type for solving future problems.

Stepsize Range and Optimal Value for Taylor–Zhang Discretization Formula Applied to Zeroing Neurodynamics Illustrated via Future Equality-Constrained Quadratic Programming

In this brief, future equality-constrained quadratic programming (FECQP) is studied. Via a zeroing neurodynamics method, a continuous-time zeroing neurodynamics (CTZN) model is presented. By using Taylor-Zhang discretization formula to discretize the CTZN model, a Taylor-Zhang discrete-time zeroing neurodynamics (TZ-DTZN) model is presented to perform FECQP. Furthermore, we focus on the critical parameter of the TZ-DTZN model, i.e., stepsize. By theoretical analyses, we obtain an effective range of the stepsize, which guarantees the stability of the TZ-DTZN model. In addition, we further discuss the optimal value of the stepsize, which makes the TZ-DTZN model possess the optimal stability (i.e., the best stability with the fastest convergence). Finally, numerical experiments and application experiments for motion generation of a robot manipulator are conducted to verify the high precision of the TZ-DTZN model and the effective range and optimal value of the stepsize for FECQP.

A Primal Neural Network for Online Equality-Constrained Quadratic Programming

This paper aims at solving online equality-constrained quadratic programming problem, which is widely encountered in science and engineering, e.g., computer vision and pattern recognition, digital signal processing, and robotics. Recurrent neural networks such as conventional GradientNet and ZhangNet are considered as powerful solvers for such a problem in light of its high computational efficiency and capability of circuit realisation. In this paper, an improved primal recurrent neural network and its electronic implementation are proposed and analysed. Compared to the existing recurrent networks, i.e. GradientNet and ZhangNet, our network can theoretically guarantee superior global exponential convergence. Robustness performance of our such neural model is also analysed under a large model implementation error, with the upper bound of stead-state solution error estimated. Simulation results demonstrate theoretical analysis on the proposed model, which also verify the effectiveness of the proposed model for online equality-constrained quadratic programming.

The ADMM Algorithm for Distributed Quadratic Problems: Parameter Selection and Constraint Preconditioning

This paper presents optimal parameter selection and preconditioning of the alternating direction method of multipliers (ADMM) algorithm for a class of distributed quadratic problems, which can be formulated as equality-constrained quadratic programming problems. The parameter selection focuses on the ADMM step-size and relaxation parameter, while the preconditioning corresponds to selecting the edge weights of the underlying communication graph. We optimize these parameters to yield the smallest convergence factor of the iterates. Explicit expressions are derived for the step-size and relaxation parameter, as well as for the corresponding convergence factor. Numerical simulations justify our results and highlight the benefits of optimal parameter selection and preconditioning for the ADMM algorithm.

Taylor O(h³) Discretization of ZNN Models for Dynamic Equality-Constrained Quadratic Programming With Application to Manipulators.

In this paper, a new Taylor-type numerical differentiation formula is first presented to discretize the continuous-time Zhang neural network (ZNN), and obtain higher computational accuracy. Based on the Taylor-type formula, two Taylor-type discrete-time ZNN models (termed Taylor-type discrete-time ZNNK and Taylor-type discrete-time ZNNU models) are then proposed and discussed to perform online dynamic equality-constrained quadratic programming. For comparison, Euler-type discrete-time ZNN models (called Euler-type discrete-time ZNNK and Euler-type discrete-time ZNNU models) and Newton iteration, with interesting links being found, are also presented. It is proved herein that the steady-state residual errors of the proposed Taylor-type discrete-time ZNN models, Euler-type discrete-time ZNN models, and Newton iteration have the patterns of O(h(3)), O(h(2)), and O(h), respectively, with h denoting the sampling gap. Numerical experiments, including the application examples, are carried out, of which the results further substantiate the theoretical findings and the efficacy of Taylor-type discrete-time ZNN models. Finally, the comparisons with Taylor-type discrete-time derivative model and other Lagrange-type discrete-time ZNN models for dynamic equality-constrained quadratic programming substantiate the superiority of the proposed Taylor-type discrete-time ZNN models once again.

Performance analysis of gradient neural network exploited for online time-varying quadratic minimization and equality-constrained quadratic programming

In this paper, the performance of a gradient neural network (GNN), which was designed intrinsically for solving static problems, is investigated, analyzed and simulated in the situation of time-varying coefficients. It is theoretically proved that the gradient neural network for online solution of time-varying quadratic minimization (QM) and quadratic programming (QP) problems could only approximately approach the time-varying theoretical solution, instead of converging exactly. That is, the steady-state error between the GNN solution and the theoretical solution can not decrease to zero. In order to understand the situation better, the upper bound of such an error is estimated firstly, and then the global exponential convergence rate is investigated for such a GNN when approaching an error bound. Computer-simulation results, including those based on a six-link robot manipulator, further substantiate the performance analysis of the GNN exploited to solve online time-varying QM and QP problems.

PLS Algorithms for Equality-Constrained Linear-Phase FIR Filter Design

The least-squares design of linear-phase finite impulse response (FIR) filters with equality constraints in the time and frequency domains can be formulated as an equality-constrained quadratic programming (QP) problem. This paper presents an effective and robust novel algorithm, the projected least-squares (PLS) algorithm, for equality-constrained QP problems. The algorithm eventually projects an unconstrained minimization solution successively onto the feasible hyperplane of the problem. An additional term is added into the Hessian matrix of the cost function, thus ensuring the positive definiteness of the Hessian matrices during the iterative procedure for the design of constrained minimax filters. To demonstrate its effectiveness and robustness, the PLS algorithm is applied to design optimal linear-phase Nyquist filters that may satisfy some frequency domain constraints besides the zero-crossing impulse response.

Error analysis of an algorithm for equality-constrained quadratic programming problems

In this paper the numerical stability of the orthogonal factorization method [5] for linear equality-constrained quadratic programming problems is studied using a backward error analysis. A perturbation formula for the problem is analyzed; the condition numbers of this formula are examined in order to compare them with the condition numbers of the two matrices of the problem. A class of test problems is also considered in order to show experimentally the behaviour of the method.

Numerical solution of equality-constrained quadratic programming problems on vector-parallel computers

This paper is concerned with the development of the method of multipliers combined with the conjugate gradient algorithm for solving the special linear system where A is a real symmetric non negative definite (n × n) matrixB is a real (n × m) matrix with full column rank (m < n) and A and BT have no nontrivial null vector in common. We assume that A and BT are large and not extremely sparse. The proposed method is well suitable for parallel implementation on a multiprocessor system that can execute concurrently different tasks on a few vector processors with shared central memory, such as the CRAY Y-MP. The results of an extensive computer experimentation, which is aimed at evaluating the effectiveness of the method, are reported.

A computer-aided-design technique for design of a six-section elliptic tire-building drum.

In this investigation, a computer-aided-design method for a six-section elliptic tire-building drum is presented instead of the traditional geometric construction. The mathematical formulation of the proposed CAD technique is discussed in some detail. The efficient recursive equality-constrained quadratic programming method for solving constrained optimization problems is introduced to synthesize the mechanism of the drum. One practical problem is solved to illustrate the method.

A fast algorithm for equality-constrained quadratic programming on the alliant FX/8

An efficient implementation of the null-space method for quadratic programming on the Alliant FX/8 computer is described. The most computationally significant operations in this method are the orthogonal factorization of the constraint matrix and corresponding similarity transformation of the Hessian, and the Cholesky factorization of the reduced Hessian matrix. It is shown how these can be implemented in such a way as to take full advantage of the Alliant's parallel/vector capabilities and memory hierarchy. Timing results are given on a set of test problems for which the data can be easily accommodated in core memory.

Local properties of inexact methods for minimizing nonsmooth composite functions

Methods for minimization of composite functions with a nondifferentiable polyhedral convex part are considered. This class includes problems involving minimax functions and norms. Local convergence results are given for "active set" methods, in which an equality-constrained quadratic programming subproblem is solved at each iteration. The active set consists of components of the polyhedral convex function which are active or near-active at the current iteration. The effects of solving the subproblem inexactly at each iteration are discussed; rate-of-convergence results which depend on the degree of inexactness are given.

A Parallel Block Iterative Scheme Applied to Computations in Structural Analysis

In this paper it is shown how a block cyclic successive overrelaxation direct-iterative method can be applied to the parallel solution of certain large-scale linear equality-constrained quadratic programming problems. The scheme is similar in nature to those studied recently by de Pillis, Niethammer and Varga and by Markham, Neumann and Plemmons for solving large sparse least squares problems. It is based upon a partitioning strategy of the fundamental matrix into a block consistently ordered 2-cyclic form where the nonzero eigenvalues of the Jacobi matrix are all pure imaginary. The method is shown to be globally convergent and convergence rates are established. Applications of the algorithm are discussed for large-scale structural analysis computations where it is shown how the algorithm can be adapted to the simultaneous computation of the system forces and the nodal displacements. Here, advantage can be taken of the special forms of the matrices involved. In particular, it is shown that much of the algorithm lends itself to efficient implementation on pipelined vector machines and on multiprocessors.

The Numerical Solution of Equality-Constrained Quadratic Programming Problems

The Numerical Solution of Equality-Constrained Quadratic Programming Problems

A Projected Lagrangian Algorithm for Nonlinear Minimax Optimization

The minimax problem is an unconstrained optimization problem whose objective function is not differentiable everywhere, and hence cannot be solved efficiently by standard techniques for unconstrained optimization. It is well known that the problem can be transformed into a nonlinearly constrained optimization problem with one extra variable, where the objective and constraint functions are continuously differentiable. This equivalent problem has special properties which are ignored if solved by a general-purpose constrained optimization method. The algorithm we present exploits the special structure of the equivalent problem. A direction of search is obtained at each iteration of the algorithm by solving an equality-constrained quadratic programming problem, related to one a projected Lagrangian method might use to solve the equivalent constrained optimization problem. Special Lagrange multiplier estimates are used to form an approximation to the Hessian of the Lagrangian function, which appears in the quadratic program. Analytical Hessians, finite differencing or quasi-Newton updating may be used in the approximation of this matrix. The resulting direction of search is guaranteed to be a descent direction for the minimax objective function. Under mild conditions the algorithms are locally quadratically convergent if analytical Hessians are used.