Convex Quadratic Optimization Problem Research Articles

Superconvergence property of finite element methods for parabolic optimal control problems

In this paper, a finite element method for a parabolic optimalcontrol problem is introduced and analyzed. For the discretizationof a quadratic convex optimal control problem, the state andco-state are approximated by piecewise linear functions and thecontrol is approximated by piecewise constant functions. As aresult, it is proved in this paper that the difference between asuitable interpolation of the control and its finite elementapproximation has superconvergence property in order $O(h^2)$.Finally, two numerical examples are presented to confirm ourtheoretical results.

Error estimates for parabolic optimal control problem by fully discrete mixed finite element methods

In this paper we study the fully discrete mixed finite element methods for quadratic convex optimal control problem governed by parabolic equations. The space discretization of the state variable is done using usual mixed finite elements, whereas the time discretization is based on difference methods. The state and the co-state are approximated by the lowest order Raviart–Thomas mixed finite element spaces and the control is approximated by piecewise constant functions. By applying some error estimates techniques of standard mixed finite element methods, we derive a priori error estimates both for the coupled state and the control approximation. Finally, we present some numerical examples which confirm our theoretical results.

Bi-parametric convex quadratic optimization

In this paper, we consider the convex quadratic optimization problem with simultaneous perturbation in the right-hand side of the constraints and the linear term of the objective function with different parameters. The regions with invariant optimal partitions as well as the behaviour of the optimal value function on the regions are investigated. We show that identifying these regions can be done in polynomial time in the output size. An algorithm for identifying all invariancy regions is presented. Some implementation details as well as a numerical example are discussed.

Smooth Flight Trajectory Planning in the Presence of No-Fly Zones and Obstacles

This paper describes a novel procedure to generate continuously differentiable optimal flight trajectories in the presence of arbitrarily shaped no-fly zones and obstacles having a fixed position in time. The operational flight scenario is first discretized with a finite dimensional grid of positions-directions pairs. A weighted and oriented graph is then defined for which the nodes are the earlier mentioned grid points and for which the arcs correspond to minimum length trajectories compliant with obstacle avoidance constraints. Arcs are obtained via solving convex quadratic programming optimization problems that can also account for geometrical constraints such as trajectory curvature limitations. The problem of finding an optimal trajectory be tween two nodes of the so-called core paths graph is then solved via a minimum cost path search algorithm. In a real-time application perspective, the generation of the core paths graph is computationally cumbersome. Moreover, the aircraft position and direction rarely coincide with one of the graph nodes. However, if the graph is built offline and stored, the definition of an optimal trajectory connecting any points of the space domain requires a reduced computational effort. The particular case of piecewise polynomial trajectories minimizing a flight path's length, compliant with constraints on curvature and flight-path angles, is fully developed. Two- and three-dimensional examples are discussed to show the applicability as well as the effectiveness of the technique.

Path Planning for Wheeled Mobile Robots Using Core Paths Graphs

This paper describes a novel procedure based on Core Path Graphs to generate continuously differentiable sub-optimal paths for wheeled mobile robots in the presence of obstacles. The operational scenario is first discretized with a finite dimensional grid of positions-directions pairs. A weighted and oriented graph is then defined whose nodes are the above mentioned grid points, and whose arcs correspond to minimum length trajectories compliant with obstacle avoidance constraints. Arcs are obtained via solving convex quadratic programming optimization problems. A minimum cost path search algorithm is then solved to find an optimal trajectory between two nodes of the so-called Core Paths Graph. The presence of a-priori unknown obstacles is managed by isolating on-line the non suitable arcs of the Core Paths Graph. A numerical example on a wheeled mobile robot is discussed to show the applicability of the proposed technique

Error estimates of fully discrete mixed finite element methods for semilinear quadratic parabolic optimal control problem

In this paper we study the fully discrete mixed finite element methods for quadratic convex optimal control problem governed by semilinear parabolic equations. The space discretization of the state variable is done using usual mixed finite elements, whereas the time discretization is based on difference methods. The state and the co-state are approximated by the lowest order Raviart–Thomas mixed finite element spaces and the control is approximated by piecewise constant elements. By applying some error estimates techniques of mixed finite element methods, we derive a priori error estimates both for the coupled state and the control approximation. Finally, we present a numerical example which confirms our theoretical results.

Gradient learning in a classification setting by gradient descent

Learning gradients is one approach for variable selection and feature covariation estimation when dealing with large data of many variables or coordinates. In a classification setting involving a convex loss function, a possible algorithm for gradient learning is implemented by solving convex quadratic programming optimization problems induced by regularization schemes in reproducing kernel Hilbert spaces. The complexity for such an algorithm might be very high when the number of variables or samples is huge. We introduce a gradient descent algorithm for gradient learning in classification. The implementation of this algorithm is simple and its convergence is elegantly studied. Explicit learning rates are presented in terms of the regularization parameter and the step size. Deep analysis for approximation by reproducing kernel Hilbert spaces under some mild conditions on the probability measure for sampling allows us to deal with a general class of convex loss functions.

Local Classifier Weighting by Quadratic Programming

It has been widely accepted that the classification accuracy can be improved by combining outputs of multiple classifiers. However, how to combine multiple classifiers with various (potentially conflicting) decisions is still an open problem. A rich collection of classifier combination procedures -- many of which are heuristic in nature -- have been developed for this goal. In this brief, we describe a dynamic approach to combine classifiers that have expertise in different regions of the input space. To this end, we use local classifier accuracy estimates to weight classifier outputs. Specifically, we estimate local recognition accuracies of classifiers near a query sample by utilizing its nearest neighbors, and then use these estimates to find the best weights of classifiers to label the query. The problem is formulated as a convex quadratic optimization problem, which returns optimal nonnegative classifier weights with respect to the chosen objective function, and the weights ensure that locally most accurate classifiers are weighted more heavily for labeling the query sample. Experimental results on several data sets indicate that the proposed weighting scheme outperforms other popular classifier combination schemes, particularly on problems with complex decision boundaries. Hence, the results indicate that local classification-accuracy-based combination techniques are well suited for decision making when the classifiers are trained by focusing on different regions of the input space.

Generalization performance of support vector machines and neural networks in runoff modeling

Effective one-day lead runoff prediction is one of the significant aspects of successful water resources management in arid region. For instance, reservoir and hydropower systems call for real-time or on-line site-specific forecasting of the runoff. In this research, we present a new data-driven model called support vector machines (SVMs) based on structural risk minimization principle, which minimizes a bound on a generalized risk (error), as opposed to the empirical risk minimization principle exploited by conventional regression techniques (e.g. ANNs). Thus, this stat-of-the-art methodology for prediction combines excellent generalization property and sparse representation that lead SVMs to be a very promising forecasting method. Further, SVM makes use of a convex quadratic optimization problem; hence, the solution is always unique and globally optimal. To demonstrate the aforementioned forecasting capability of SVM, one-day lead stream flow of Bakhtiyari River in Iran was predicted using the local climate and rainfall data. Moreover, the results were compared with those of ANN and ANN integrated with genetic algorithms (ANN-GA) models. The improvements in root mean squared error (RMSE) and squared correlation coefficient (R2) by SVM over both ANN models indicate that the prediction accuracy of SVM is at least as good as that of those models, yet in some cases actually better, as well as forecasting of high-value discharges.

Superconvergence of quadratic optimal control problems by triangular mixed finite element methods

AbstractThe aim of this work is to investigate the discretization of a quadratic convex optimal control problem using the mixed finite element method. The state and co‐state are approximated by the order k⩽1 Raviart–Thomas mixed finite element spaces, and the control is approximated by piecewise constant functions. We construct an interpolation of the exact control and a projection of the discrete scalar co‐state to be the approximated solution of the control variable for the continuous optimal control problem. As a result, it can be proved that the difference between the interpolation and the piecewise constant approximation has superconvergence property for the control of order h3/2 for k=0 and of order h2 for k=1. Moreover, only for the order k=1 Raviart–Thomas mixed finite element approximation does the postprocessing technique possess the superconvergence property of order h2. Finally, some numerical examples are given to demonstrate the practical side of the theoretical results about superconvergence. Copyright © 2008 John Wiley & Sons, Ltd.

Self-adaptive support vector machines: modelling and experiments

Method In this paper, we introduce a bi-level optimization formulation for the model and feature selection problems of support vector machines (SVMs). A bi-level optimization model is proposed to select the best model, where the standard convex quadratic optimization problem of the SVM training is cast as a subproblem.

Superconvergence of mixed finite element methods for optimal control problems

In this paper, we investigate the superconvergence property of the numerical solution of a quadratic convex optimal control problem by using rectangular mixed finite element methods. The state and co-state variables are approximated by the lowest order Raviart-Thomas mixed finite element spaces and the control variable is approximated by piecewise constant functions. Some realistic regularity assumptions are presented and applied to error estimation by using an operator interpolation technique. We derive $L^2$ superconvergence properties for the flux functions along the Gauss lines and for the scalar functions at the Gauss points via mixed projections. Moreover, global $L^2$ superconvergence results are obtained by virtue of an interpolation postprocessing technique. Thus, based on these superconvergence estimates, some asymptotic exactness a posteriori error estimators are presented for the mixed finite element methods. Finally, some numerical examples are given to demonstrate the practical side of the theoretical results about superconvergence.

Robust Adaptive Beamforming Using Sequential Quadratic Programming: An Iterative Solution to the Mismatch Problem

A new approach to the design of robust adaptive beamforming is introduced. The essence of the new approach is to estimate the difference between the actual and presumed steering vectors and to use this difference to correct the erroneous presumed steering vector. The estimation process is performed iteratively where a quadratic convex optimization problem is solved at each iteration. Contrary to the worst-case performance-based and the probability-constrained-based approaches, our approach does not make any assumptions on either the norm of the mismatch vector or its probability distribution. Hence, it avoids the need for estimating their values.

Simultaneous On-Chip Bus Synthesis and Voltage Scaling Under Random On-Chip Data Traffic

This paper presents a statistical approach to synthesize an energy conscious the optimal bus width and the number of buses. The slack is exploited to maximize bus sharing and to reduce energy consumption by simultaneously scaling the voltage during the synthesis of on-chip communication bus. An assumption for bus synthesis is that a system has been partitioned and mapped onto the appropriate modules of a system-on-chip (SoC). Because of the diversity of applications to be run on a single SoC, there exists a variability of data size to be transferred among the on-chip communicating modules. This variability of data size is modeled as a random variable with a known distribution function. The resulting synthesis problem is relaxed to a convex quadratic optimization problem and solved efficiently using a convex optimization tool. The effectiveness of our approach is demonstrated by applying optimization to an automatically generated benchmark and a real-life application. By scaling voltage of a bus, a tradeoff between communication bus cost (bus width and the number of buses) and energy reduction is explored. The experimental results show the significant reduction in communication energy with scaling voltage. However, it offers a limitation to minimize the communication bus cost, if the voltage is scaled beyond its minimum limit. Furthermore, we also estimate the distribution of voltage under a random data size using an analytical method and the Monte Carlo simulation. The results show that the analytically estimated statistical parameters of voltage are close to the simulated results.

Optimal Resource Allocation Scheme in a Multirate Overlapped Optical CDMA System

This paper addresses the problem of resource allocation in a multiservice optical network based on an overlapped code-division-multiple-access system. A joint transmission power and overlapping coefficient (transmission rate) allocation strategy is provided via the solution of a constrained convex quadratic optimization problem. The solution of this problem maximizes the aggregate throughput subject to peak laser transmission power constraints. The optimization problem is solved in a closed form, and the resource allocation strategy is simple to implement in an optical network. Simulation results are presented, showing a total agreement between the derived analytical solution and the one obtained using a numerical search method. In addition, analytical and numerical results show that the proposed resource allocation strategy can offer substantial improvement in the system throughput.

Support set expansion sensitivity analysis in convex quadratic optimization

In support set expansion sensitivity analysis, one concerns to find the range of parameter variation where the perturbed problem has an optimal solution with the support set that includes the support set of the given optimal solution of the unperturbed problem. In this article, we consider the perturbed convex quadratic optimization problem and present a method to identify the support set expansion sets for this problem.

Active Constraint Set Invariancy Sensitivity Analysis in Linear Optimization

Active constraint set invariancy sensitivity analysis is concerned with finding the range of parameter variation so that the perturbed problem has still an optimal solution with the same support set that the given optimal solution of the unperturbed problem has. However, in an optimization problem with inequality constraints, active constraint set invariancy sensitivity analysis aims to find the range of parameter variation, where the active constraints in a given optimal solution remains invariant. For the sake of simplicity, we consider the primal problem in standard form and consequently its dual may have an optimal solution with some active constraints. In this paper, the following question is answered: "what is the range of the parameter, where for each parameter value in this range, a dual optimal solution exists with exactly the same set of positive slack variables as for the current dual optimal solution?". The differences of the results between the linear and convex quadratic optimization problems are highlighted too.

On the complexity of working set selection

The decomposition method is currently one of the major methods for solving the convex quadratic optimization problems being associated with Support Vector Machines (SVM-optimization). A key issue in this approach is the policy for working set selection. We would like to find policies that realize (as well as possible) three goals simultaneously: “(fast) convergence to an optimal solution”, “efficient procedures for working set selection”, and “a high degree of generality” (including typical variants of SVM-optimization as special cases). In this paper, we study a general policy for working set selection that has been proposed in [Nikolas List, Hans Ulrich Simon, A general convergence theorem for the decomposition method, in: Proceedings of the 17th Annual Conference on Computational Learning Theory, 2004, pp. 363–377] and further analyzed in [Nikolas List, Hans Ulrich Simon, General polynomial time decomposition algorithms, in: Proceedings of the 17th Annual Conference on Computational Learning Theory, 2005, pp. 308–322]. It is known that it efficiently approaches feasible solutions with minimum cost for any convex quadratic optimization problem. Here, we investigate its computational complexity when it is used for SVM-optimization. It turns out that, for a variable size of the working set, the general policy poses an NP-hard working set selection problem. But a slight variation of it (sharing the convergence properties with the original policy) can be solved in polynomial time. For working sets of fixed size 2, the situation is even better. In this case, the general policy coincides with the “rate certifying pair approach” (introduced by Hush and Scovel). We show that maximum rate certifying pairs can be found in linear time, which leads to a quite efficient decomposition method with a polynomial convergence rate for SVM-optimization.

Generalized support set invariancy sensitivity analysis in linear optimization

Support set invariancy sensitivity analysis deals with finding the range of the parameter variation where there are optimal solutions with the same positive variables for all parameter values throughout this range. This approach to sensitivity analysis has been studied for Linear Optimization (LO) and Convex Quadratic Optimization (CQO) problems, when they are in standard form. In practice, most problems are in <em>general form</em>, in addition to nonnegative variables and equalities, they include free variables and inequalities. The LO problem in general form can be converted into the standard form, but this transforming changes the meaning of the support set invariancy sensitivity analysis. <br>&nbsp; &nbsp In this paper, we consider the primal and dual LO problems in general form and introduce the associated general standard form. It is shown that investigating support set invariancy sensitivity analysis for this general standard form is able to accommodate not only the support set invariancy sensitivity analysis for usual standard form, but also the classic study of sensitivity analysis based on simplex methods as well as the recent point of view of sensitivity analysis based on interior point methods.

Sensitivity analysis in convex quadratic optimization: invariant support set interval

In sensitivity analysis one wants to know how the problem and the optimal solutions change under the variation of the input data. We consider the case when variation happens in the right-hand side of the constraints and/or in the linear term of the objective function. We are interested to find the range of the parameter variation in Convex Quadratic Optimization (CQO) problems where the support set of a given primal optimal solution remains invariant. This question has been first raised in Linear Optimization (LO) and known as Type II (so-called Support Set Invariancy) sensitivity analysis. We present computable auxiliary problems to identify the range of parameter variation in support set invariancy sensitivity analysis for CQO. It should be mentioned that all the given auxiliary problems are LO problems and can be solved by an interior point method in polynomial time. We also highlight the differences between the characteristics of support set invariancy sensitivity analysis for LO and CQO.