General Linear Constraints Research Articles

A dual gradient-projection method for large-scale strictly convex quadratic problems

The details of a solver for minimizing a strictly convex quadratic objective function subject to general linear constraints are presented. The method uses a gradient projection algorithm enhanced with subspace acceleration to solve the bound-constrained dual optimization problem. Such gradient projection methods are well-known, but are typically employed to solve the primal problem when only simple bound-constraints are present. The main contributions of this work are threefold. First, we address the challenges associated with solving the dual problem, which is usually a convex problem even when the primal problem is strictly convex. In particular, for the dual problem, one must efficiently compute directions of infinite descent when they exist, which is precisely when the primal formulation is infeasible. Second, we show how the linear algebra may be arranged to take computational advantage of sparsity that is often present in the second-derivative matrix, mostly by showing how sparse updates may be performed for algorithmic quantities. We consider the case that the second-derivative matrix is explicitly available and sparse, and the case when it is available implicitly via a limited memory BFGS representation. Third, we present the details of our Fortran 2003 software package DQP, which is part of the GALAHAD suite of optimization routines. Numerical tests are performed on quadratic programming problems from the combined CUTEst and Maros and Meszaros test sets.

Optimal portfolio selection with maximal risk adjusted return

ABSTRACTWe investigate the portfolio diversification problem by maximizing the risk adjusted return (RAR) of the underlying portfolio. The model in this article has two primary advantages over the original portfolio selection model with maximal RAR: (1) it considers the set of available assets containing any number of assets instead of only two assets, which is more reasonable in practical applications and (2) it incorporates the general linear constraint other than the simple budget constraint, which can deal with additional constraints for rational investors. An application including in-sample and out-of-sample tests is provided where the results illustrate that the portfolios selected by our method lead to considerable increases of RAR in comparison with those by the minimization of variance approach, and the outperformance persists using different sample frequencies.

Deformable Objects Collision Handling with Fast Convergence

We present a stable and efficient simulator for deformable objects with collisions and contacts. For stability, an optimization derived from the implicit time integrator is solved in each timestep under the inequality constraints coming from collisions. To achieve fast convergence, we extend the MPRGP based solver from handling box constraints only to handling general linear constraints and prove its convergence. This generalization introduces a cost of solving dense linear systems in each step, but these systems can be reduced into diagonal ones for efficiency without affecting the general stability via pruning redundant collisions. Our solver is an order of magnitude faster, especially for elastic objects under large deformation compared with iterative constraint anticipation method ICA, a typical method for stability. The efficiency, robustness and stability are further verified by our results.

Español

This paper presents a software tool running on WindowsR operating system, intended for formal and systematic design of supervisory controllers for discrete event systems described by Petri nets, and the automatic generation of Ladder code for RsLogix 500TM software. The software is based on general linear vector constraints, and in case of the inadmissibility of some of the imposed constraints, the software has three methods to transform them in admissible ones. The software has also two methods to find siphons and traps on the supervised PN, which is useful to prevent the dead lock of the supervised system. Once the supervisor fulfills the control requirements, the tool can transfer the supervisor, automatically, to Ladder code by the Token Passing Logic extended method.

On fast trust region methods for quadratic models with linear constraints

Abstract Quadratic models $$Q_k ( \underline{x}), \underline{x}\in \mathcal{R}^n$$ Q k ( x ̲ ) , x ̲ ∈ R n , of the objective function $$F ( \underline{x}), \underline{x}\in \mathcal{R}^n$$ F ( x ̲ ) , x ̲ ∈ R n , are used by many successful iterative algorithms for minimization, where k is the iteration number. Given the vector of variables $$\underline{x}_k \in \mathcal{R}^n$$ x ̲ k ∈ R n , a new vector $$\underline{x}_{k+1}$$ x ̲ k + 1 may be calculated that satisfies $$Q_k ( \underline{x}_{k+1} ) < Q_k ( \underline{x}_k )$$ Q k ( x ̲ k + 1 ) < Q k ( x ̲ k ) , in the hope that it provides the reduction $$F ( \underline{x}_{k+1} ) < F ( \underline{x}_k )$$ F ( x ̲ k + 1 ) < F ( x ̲ k ) . Trust region methods include a bound of the form $$\Vert \underline{x}_{k+1} - \underline{x}_k \Vert \le \Delta _k$$ ‖ x ̲ k + 1 - x ̲ k ‖ ≤ Δ k . Also we allow general linear constraints on the variables that have to hold at $$\underline{x}_k$$ x ̲ k and at $$\underline{x}_{k+1}$$ x ̲ k + 1 . We consider the construction of $$\underline{x}_{k+1}$$ x ̲ k + 1 , using only of magnitude $$n^2$$ n 2 operations on a typical iteration when n is large. The linear constraints are treated by active sets, which may be updated during an iteration, and which decrease the number of degrees of freedom in the variables temporarily, by restricting $$\underline{x}$$ x ̲ to an affine subset of $$\mathcal{R}^n$$ R n . Conjugate gradient and Krylov subspace methods are addressed for adjusting the reduced variables, but the resultant steps are expressed in terms of the original variables. Termination conditions are given that are intended to combine suitable reductions in $$Q_k ( \cdot )$$ Q k ( · ) with a sufficiently small number of steps. The reason for our work is that $$\underline{x}_{k+1}$$ x ̲ k + 1 is required in the LINCOA software of the author, which is designed for linearly constrained optimization without derivatives when there are hundreds of variables. Our studies go beyond the final version of LINCOA, however, which employs conjugate gradients with termination at the trust region boundary. In particular, we find that, if an active set is updated at a point that is not the trust region centre, then the Krylov method may terminate too early due to a degeneracy. An extension to the conjugate gradient method for searching round the trust region boundary receives attention too, but it was also rejected from LINCOA, because of poor numerical results. The given techniques of LINCOA seem to be adequate in practice.

Shape restricted smoothing splines via constrained optimal control and nonsmooth Newton’s methods

Shape restricted smoothing splines receive considerable attention, motivated by many important applications in science and engineering. In this paper, we consider smoothing splines subject to general linear dynamics and control constraints, and formulate them as finite-horizon constrained linear optimal control problems with unknown initial state and control. By exploring techniques from functional and variational analyses, optimality conditions are developed in terms of variational inequalities. Due to the control constraints, the optimality conditions give rise to a nonsmooth B-differentiable equation of an optimal initial condition, whose unique solution completely determines the shape restricted smoothing spline. A modified nonsmooth Newton’s algorithm with line search is used to solve this equation; detailed convergence analysis of the proposed algorithm is presented. Using techniques from nonsmooth analysis and polyhedral theory, we show the global convergence of the algorithm for shape restricted smoothing splines subject to general polyhedral control constraints.

Advantages of simplicial partitioning for Lipschitz optimization problems with linear constraints

The well known DIRECT (DIviding RECTangles) algorithm for global optimization requires bound constraints on variables and does not naturally address additional linear or nonlinear constraints. A feasible region defined by linear constraints may be covered by simplices, therefore simplicial partitioning may tackle linear constraints in a very subtle way. In this paper we demonstrate this advantage of simplicial partitioning by applying a recently proposed deterministic simplicial partitions based DISIMPL algorithm for optimization problems defined by general linear constraints (Lc-DISIMPL). An extensive experimental investigation reveals advantages of this approach to such problems comparing with different constraint-handling methods, proposed for use with DIRECT. Furthermore the Lc-DISIMPL algorithm gives very competitive results compared to a derivative-free particle swarm algorithm (PSwarm) which was previously shown to give very promising results. Moreover, DISIMPL guarantees the convergence to the global solution, whereas the PSwarm algorithm sometimes fails to converge to the global minimum.

Improvement of the convergence speed of a discrete-time recurrent neural network for quadratic optimization with general linear constraints

In this work a specific preconditioning technique is developed to improve the convergence speed of a discrete-time recurrent neural network for quadratic optimization with general linear constraints. The discrete-time network is a model recently published with the broadest range of applicability to various optimization problems and constraints. The proposed preconditioning technique is shown to improve the convergence speed of the model significantly, and thus contribute to enhance the application of the model in these problems. In addition to the theoretical analysis, extensive experimental results are presented to illustrate the technique developed, and to show the significant improvement attained.

A neural network model for solving convex quadratic programming problems with some applications

This paper presents a capable neural network for solving strictly convex quadratic programming (SCQP) problems with general linear constraints. The proposed neural network model is stable in the sense of Lyapunov and can converge to an exact optimal solution of the original problem. A block diagram of the proposed model is also given. Several applicable examples further show the correctness of the results and the good performance of the model.

On Wolfe's Method for Resolving Degeneracy in Linearly Constrained Optimization

Wolfe [J. Soc. Indust. Appl. Math., 11 (1963), pp. 205--211] describes a novel and very useful method for resolving degeneracy in the Simplex method for Linear Programming (LP). The simplicity and reliability of the method makes it an excellent choice in this author's opinion. The method is described here in the context of an active set method (ASM) format for LP. The method solves recursively generated subproblems that are smaller than the original, which contributes to efficiency. Data structures are described that enable the recursive aspect of the method to be implemented very easily with minimal storage overhead. The method is extended to solve a general form of linearly constrained optimization problem that includes quadratic programming (QP) and allows simple bounds on the variables and both equation and inequality general linear constraints. Issues of round-off error are discussed and heuristics are described that have proved to be very reliable in practice. The method is further extended to QP problems in which general linear constraints are handled as $L_1$ terms in the objective function.

Laplace inversion of low‐resolution NMR relaxometry data using sparse representation methods

Low-resolution nuclear magnetic resonance (LR-NMR) relaxometry is a powerful tool that can be harnessed for characterizing constituents in complex materials. Conversion of the relaxation signal into a continuous distribution of relaxation components is an ill-posed inverse Laplace transform problem. The most common numerical method implemented today for dealing with this kind of problem is based on L2-norm regularization. However, sparse representation methods via L1 regularization and convex optimization are a relatively new approach for effective analysis and processing of digital images and signals. In this article, a numerical optimization method for analyzing LR-NMR data by including non-negativity constraints and L1 regularization and by applying a convex optimization solver PDCO, a primal-dual interior method for convex objectives, that allows general linear constraints to be treated as linear operators is presented. The integrated approach includes validation of analyses by simulations, testing repeatability of experiments, and validation of the model and its statistical assumptions. The proposed method provides better resolved and more accurate solutions when compared with those suggested by existing tools. © 2013 Wiley Periodicals, Inc. Concepts Magn Reson Part A 42A: 72–88, 2013.

Computing efficient exact designs of experiments using integer quadratic programming

A new method for computing exact experimental designs for linear regression models by integer quadratic programming is proposed. The key idea is to use the criterion of DQ-optimality, which is a quadratic approximation of the criterion of D-optimality in the neighbourhood of the approximate D-optimal information matrix. Several numerical examples are used to demonstrate that the D-efficiency of exact DQ-optimal designs is usually very high. An important advantage of this method is that it can be applied to situations with general linear constraints on permissible designs, including marginal and cost constraints.

New discrete-time recurrent neural network proposal for quadratic optimization with general linear constraints.

In this brief, the quadratic problem with general linear constraints is reformulated using the Wolfe dual theory, and a very simple discrete-time recurrent neural network is proved to be able to solve it. Conditions that guarantee global convergence of this network to the constrained minimum are developed. The computational complexity of the method is analyzed, and experimental work is presented that shows its high efficiency.

Penalty function-based precoding for downlink multiuser MIMO systems

In this paper, a novel linear precoding scheme is proposed for downlink multiuser multiple-input multiple-output (MIMO) systems. The new algorithm uses the penalty function method to mitigate the co-channel interference and is formulated as a convex problem with general linear constraints. The constraints can be sum power, per-antenna power or per-antenna-group power constraints, hence the new algorithm is general and can be used in both single-cell and fully cooperative multi-cell scenarios. Moreover, the famous block diagonalization (BD) precoding can be considered as a special case of our method when a very large penalty factor is used. We study the optimal solution of this convex problem and propose an iterative algorithm to obtain the optimum based on the Lagrange dual method. Simulation results show that the proposed method significantly outperforms the BD method at low and moderate SNR values in terms of the weighted sum rate.

Joint MMSE Transceiver Designs and Performance Benchmark for CoMP Transmission and Reception

Coordinated Multipoint (CoMP) transmission and reception has been suggested as a key enabling technology of future cellular systems. To understand different CoMP configurations and to facilitate the configuration selection (and thus determine channel state information (CSI) feedback and data sharing requirements), performance benchmarks are needed to show what performance gains are possible. A unified approach is also needed to enable the cluster of cooperating cells to systematically take care of the transceiver design. To address these needs, the generalized iterative approach (GIA) is proposed as a unified approach for the minimum mean square error (MMSE) transceiver design of general multiple-transmitter multiple-receiver multiple-input-multiple-output (MIMO) systems subject to general linear power constraints. Moreover, the optimum decoder covariance optimization approach is proposed for downlink systems. Their optimality and relationships are established and shown numerically. Five CoMP configurations (Joint Processing-Equivalent Uplink, Joint Processing-Equivalent Downlink, Joint Processing-Equivalent Single User, Noncoordinated Multipoint, and Coordinated Beamforming) are studied and compared numerically. Physical insights, performance benchmarks, and some guidelines for CoMP configuration selection are presented.

Generic Optimization of Linear Precoding in Multibeam Satellite Systems

Multibeam satellite systems have been employed to provide interactive broadband services to geographical areas under-served by terrestrial infrastructure. In this context, this paper studies joint multiuser linear precoding design in the forward link of fixed multibeam satellite systems. We provide a generic optimization framework for linear precoding design to handle any objective functions of data rate with general linear and nonlinear power constraints. To achieve this, an iterative algorithm which optimizes the precoding vectors and power allocation alternatingly is proposed and most importantly, the proposed algorithm is proved to always converge. The proposed optimization algorithm is also applicable to nonlinear dirty paper coding. As a special case, a more efficient algorithm is devised to find the optimal solution to the problem of maximizing the proportional fairness among served users. In addition, the aforementioned problems and algorithms are extended to the case that each terminal has multiple co-polarization or dual-polarization antennas. Simulation results demonstrate substantial performance improvement of the proposed schemes over conventional multibeam satellite systems, zero-forcing and regularized zero-forcing precoding schemes in terms of meeting the traffic demand, e.g., using real beam patterns, over twice higher throughput can be achieved compared with the conventional scheme. The performance of the proposed linear precoding scheme is also shown to be very close to the dirty paper coding.

Data-Based Adaptive Estimation in an Investment Model

We consider improved estimation strategies for the parameters in a capital asset pricing model under a general linear constraint; we suggest candidate subspace, a preliminary test, and shrinkage estimators. We develop a large sample theory for the estimators that include derivation of asymptotic bias and asymptotic distributional risk of the suggested estimators. The asymptotic results demonstrate the superiority of the suggested estimation technique. A simulation study shows that the method suggested here has sound finite sample properties and strongly corroborates with the theoretical result of the article. A data example is also presented to illustrate the suggested estimation strategies.

Interference Alignment: A Building Block of Coordinated Beamforming Transceiver Designs

Interference alignment (IA) has recently been used for the transceiver design of the K-user constant MIMO interference channel. Since this system is a coordinated beamforming system and there are other design criteria, it is important to evaluate IA transceiver designs in light of other designs. In this paper, relationships between IA and the MMSE, Max Min SINR, Max Sum Capacity, and Min BER transceiver designs are rigorously established for all SNR’s and power constraints. A MMSE based transceiver design approach, applicable for general linear equality power constraints, is also proposed—the first such design for this system. It is used, along with some IA transceiver designs, to demonstrate numerically the derived relationships and to perform a comparison of the designs. One of the main conclusions is that IA is a building block of the traditional transceiver designs considered (MMSE, Max Min SINR, etc.) and generally should not be pursued alone.

Estimation strategies for the regression coefficient parameter matrix in multivariate multiple regression

We consider improved estimation strategies for the parameter matrix in multivariate multiple regression under a general and natural linear constraint. In the context of two competing models where one model includes all predictors and the other restricts variable coefficients to a candidate linear subspace based on prior information, there is a need of combining two estimation techniques in an optimal way. In this scenario, we suggest some shrinkage estimators for the targeted parameter matrix. Also, we examine the relative performances of the suggested estimators in the direction of the subspace and candidate subspace restricted type estimators. We develop a large sample theory for the estimators including derivation of asymptotic bias and asymptotic distributional risk of the suggested estimators. Furthermore, we conduct Monte Carlo simulation studies to appraise the relative performance of the suggested estimators with the classical estimators. The methods are also applied on a real data set for illustrative purposes.

A Riccati approach for constrained linear quadratic optimal control

An active-set method is proposed for solving linear quadratic optimal control problems subject to general linear inequality path constraints including mixed state-control and state-only constraints. A Riccati-based approach is developed for efficiently solving the equality constrained optimal control subproblems generated during the procedure. The solution of each subproblem requires computations that scale linearly with the horizon length. The algorithm is illustrated with numerical examples.