Polyhedral Constraints Research Articles

On the risk of convex-constrained least squares estimators under misspecification

We consider the problem of estimating the mean of a noisy vector. When the mean lies in a convex constraint set, the least squares projection of the random vector onto the set is a natural estimator. Properties of the risk of this estimator, such as its asymptotic behavior as the noise tends to zero, have been well studied. We instead study the behavior of this estimator under misspecification, that is, without the assumption that the mean lies in the constraint set. For appropriately defined notions of risk in the misspecified setting, we prove a generalization of a low noise characterization of the risk due to [Found. Comput. Math. 16 (2016) 965–1029] in the case of a polyhedral constraint set. An interesting consequence of our results is that the risk can be much smaller in the misspecified setting than in the well-specified setting. We also discuss consequences of our result for isotonic regression.

Primal–Dual and Dual-Fitting Analysis of Online Scheduling Algorithms for Generalized Flow-Time Problems

We study online scheduling problems on a single processor that can be viewed as extensions of the well-studied problem of minimizing total weighted flow time. In particular, we provide a framework of analysis that is derived by duality properties, does not rely on potential functions and is applicable to a variety of scheduling problems. A key ingredient in our approach is bypassing the need for "black-box" rounding of fractional solutions, which yields improved competitive ratios. We begin with an interpretation of Highest-Density-First (HDF) as a primal-dual algorithm, and a corresponding proof that HDF is optimal for total fractional weighted flow time (and thus scalable for the integral objective). Building upon the salient ideas of the proof, we show how to apply and extend this analysis to the more general problem of minimizing $\sum_j w_j g(F_j)$, where $w_j$ is the job weight, $F_j$ is the flow time and $g$ is a non-decreasing cost function. Among other results, we present improved competitive ratios for the setting in which $g$ is a concave function, and the setting of same-density jobs but general cost functions. We further apply our framework of analysis to online weighted completion time with general cost functions as well as scheduling under polyhedral constraints.

Solution uniqueness of convex piecewise affine functions based optimization with applications to constrained ℓ1 minimization

In this paper, we study the solution uniqueness of an individual feasible vector of a class of convex optimization problems involving convex piecewise affine functions and subject to general polyhedral constraints. This class of problems incorporates many important polyhedral constrained ℓ1 recovery problems arising from sparse optimization, such as basis pursuit, LASSO, and basis pursuit denoising, as well as polyhedral gauge recovery. By leveraging the max-formulation of convex piecewise affine functions and convex analysis tools, we develop dual variables based necessary and sufficient uniqueness conditions via simple and yet unifying approaches; these conditions are applied to a wide range of ℓ1 minimization problems under possible polyhedral constraints. An effective linear program based scheme is proposed to verify solution uniqueness conditions. The results obtained in this paper not only recover the known solution uniqueness conditions in the literature by removing restrictive assumptions but also yield new uniqueness conditions for much broader constrained ℓ1-minimization problems.

Solving bilevel problems with polyhedral constraint set

Solving bilevel problems with polyhedral constraint set

Solving Reverse Convex Programming Problems Using a Generalized Cutting-Plane Method

It is well known that each convex function $$f:{\mathbb{R}}^n \longrightarrow {\mathbb{R}}$$ is supremally generated by affine functions. More precisely, each convex function $$f:{\mathbb{R}}^n \longrightarrow {\mathbb{R}}$$ is the upper envelope of its affine minorants. In this paper, we propose an algorithm for solving reverse convex programming problems by using such a representation together with a generalized cutting-plane method. Indeed, by applying this representation, we solve a sequence of problems with a smaller feasible set, in which the reverse convex constraint is replaced by a still reverse convex but polyhedral constraint. Moreover, we prove that the proposed algorithm converges, under suitable assumptions, to an optimal solution of the original problem. This algorithm is coded in MATLAB language and is evaluated by some numerical examples.

On the ℓ-Attainability Sets of Continuous Discrete Functional Differential Systems

Abstract For a functional differential system with continuous and discrete times, the problem of control with respect to an on-target vector-functional l is considered. For the case of polyhedral constraints concerning control, the notion of the l-attainability is introduced and relationships to describe this set are derived.

ДОСТИЖИМЫЕ ЗНАЧЕНИЯ ЦЕЛЕВЫХ ФУНКЦИОНАЛОВ ДЛЯ ФУНКЦИОНАЛЬНО-ДИФФЕРЕНЦИАЛЬНОЙ СИСТЕМЫ С ИМПУЛЬСНЫМ ВОЗДЕЙСТВИЕМ

For a linear functional differential system with aftereffect and impulses, a description of the attainability set is given. The attainability is considered in the term of a given system of on-target functionals in the case of polyhedral constraints with respect to control and impulses.

Facial Reduction and Partial Polyhedrality

We present FRA-Poly, a facial reduction algorithm (FRA) for conic linear programs that is sensitive to the presence of polyhedral faces in the cone. The main goals of FRA and FRA-Poly are the same, i.e., finding the minimal face containing the feasible region and detecting infeasibility, but FRA-Poly treats polyhedral constraints separately. This idea enables us to reduce the number of iterations drastically when there are many linear inequality constraints. The worst case number of iterations for FRA-poly is written in the terms of a "distance to polyhedrality" quantity and provides better bounds than FRA under mild conditions. In particular, in the case of the doubly nonnegative cone, FRA-Poly gives a worst case bound of $n$ whereas the classical FRA is $\mathcal{O}(n^2)$. Of possible independent interest, we prove a variant of Gordan-Stiemke's Theorem and a proper separation theorem that takes into account partial polyhedrality. We provide a discussion on the optimal facial reduction strategy and an instance that forces FRAs to perform many steps. We also present a few applications. In particular, we will use FRA-poly to improve the bounds recently obtained by Liu and Pataki on the dimension of certain affine subspaces which appear in weakly infeasible problems.

Improvements on Interpolation Techniques based on Linear Programming for Constrained Control

We present some improvements on interpolation based control (IC) for linear discrete-time systems with polyhedral constraints on the control and the states variables. The plant may be uncertain, time-varying, and subject to bounded disturbances. Roughly speaking, IC approach is based on the interpolation between an inner and an outer controller. The extension presented here is twofold. On the one hand, the IC approach is extended to deal with higher order inner controllers, making it e.g. possible to incorporate integral action in a vicinity of the origin. On the other hand, this paper presents a modification that better exploits the control signal range. As with previous IC strategies, the presented method demands relatively low online computational resources, and is applicable also to uncertain plants. The benefits of the improved scheme are illustrated in some examples.

Automatic Synthesis of Switching Controllers for Linear Hybrid Systems

We consider the problem of computing the controllable region of a Linear Hybrid Automaton with controllable and uncontrollable transitions, w.r.t. a reachability objective. We provide an algorithm for the finite-horizon version of the problem, based on computing the set of states that must reach a given non-convex polyhedron while avoiding another one, subject to a polyhedral constraint on the slope of the trajectory. Experimental results are presented, based on an implementation of the proposed algorithm on top of the tool SpaceEx.

Deterministic solution approach for some classes of nonlinear multilevel programs with multiple followers

In this paper we investigate multilevel programming problems with multiple followers in each hierarchical decision level. It is known that such type of problems are highly non-convex and hard to solve. A solution algorithm have been proposed by reformulating the given multilevel program with multiple followers at each level that share common resources into its equivalent multilevel program having single follower at each decision level. Even though, the reformulated multilevel optimization problem may contain non-convex terms at the objective functions at each level of the decision hierarchy, we applied multi-parametric branch-and-bound algorithm to solve the resulting problem that has polyhedral constraints. The solution procedure is implemented and tested for a variety of illustrative examples.

An active set algorithm for nonlinear optimization with polyhedral constraints

A polyhedral active set algorithm PASA is developed for solving a nonlinear optimization problem whose feasible set is a polyhedron. Phase one of the algorithm is the gradient projection method, while phase two is any algorithm for solving a linearly constrained optimization problem. Rules are provided for branching between the two phases. Global convergence to a stationary point is established, while asymptotically PASA performs only phase two when either a nondegeneracy assumption holds, or the active constraints are linearly independent and a strong second-order sufficient optimality condition holds.

Exact Post-Selection Inference for Sequential Regression Procedures

ABSTRACTWe propose new inference tools for forward stepwise regression, least angle regression, and the lasso. Assuming a Gaussian model for the observation vector y, we first describe a general scheme to perform valid inference after any selection event that can be characterized as y falling into a polyhedral set. This framework allows us to derive conditional (post-selection) hypothesis tests at any step of forward stepwise or least angle regression, or any step along the lasso regularization path, because, as it turns out, selection events for these procedures can be expressed as polyhedral constraints on y. The p-values associated with these tests are exactly uniform under the null distribution, in finite samples, yielding exact Type I error control. The tests can also be inverted to produce confidence intervals for appropriate underlying regression parameters. The R package selectiveInference, freely available on the CRAN repository, implements the new inference tools described in this article. Supplementary materials for this article are available online.

Continuity Results and Error Bounds on Pseudomonotone Vector Variational Inequalities via Scalarization

Continuity (both lower and upper semicontinuities) results of the Pareto/efficient solution mapping for a parametric vector variational inequality with a polyhedral constraint set are established via scalarization approaches, within the framework of strict pseudomonotonicity assumptions. As a direct application, the continuity of the solution mapping to a parametric weak Minty vector variational inequality is also discussed. Furthermore, error bounds for the weak vector variational inequality in terms of two known regularized gap functions are also obtained, under strong pseudomonotonicity assumptions.

A minimization algorithm for equilibrium problems with polyhedral constraints

Our aim in this paper is to study an infeasible interior proximal method for solving equilibrium problems with polyhedral constrains in a quasiconvex setting. Under mild assumptions, we show that the sequence generated by our algorithm is well defined and it converges to a solution of the problem when regularization parameters converge to zero.

A branch-and-bound multi-parametric programming approach for non-convex multilevel optimization with polyhedral constraints

In this paper we develop a general but smooth global optimization strategy for nonlinear multilevel programming problems with polyhedral constraints. At each decision level successive convex relaxations are applied over the non-convex terms in combination with a multi-parametric programming approach. The proposed algorithm reaches the approximate global optimum in a finite number of steps through the successive subdivision of the optimization variables that contribute to the non-convexity of the problem and partitioning of the parameter space. The method is implemented and tested for a variety of bilevel, trilevel and fifth level problems which have non-convexity formulation at their inner levels.

Polyhedral Clinching Auctions and the AdWords Polytope

A central issue in applying auction theory in practice is the problem of dealing with budget-constrained agents. A desirable goal in practice is to design incentive compatible, individually rational, and Pareto optimal auctions while respecting the budget constraints. Achieving this goal is particularly challenging in the presence of nontrivial combinatorial constraints over the set of feasible allocations. Toward this goal and motivated by AdWords auctions, we present an auction for polymatroidal environments satisfying these properties. Our auction employs a novel clinching technique with a clean geometric description and only needs an oracle access to the submodular function defining the polymatroid. As a result, this auction not only simplifies and generalizes all previous results, it applies to several new applications including AdWords Auctions, bandwidth markets, and video on demand. In particular, our characterization of the AdWords auction as polymatroidal constraints might be of independent interest. This allows us to design the first mechanism for Ad Auctions taking into account simultaneously budgets, multiple keywords and multiple slots. We show that it is impossible to extend this result to generic polyhedral constraints. This also implies an impossibility result for multiunit auctions with decreasing marginal utilities in the presence of budget constraints.

Global Error Bounds for Systems of Convex Polynomials over Polyhedral Constraints

This paper is devoted to studying the Lipschitzian/Hölderian-type global error bound for systems of finitely many convex polynomial inequalities over a polyhedral constraint. First, for systems of this type, we show that under a suitable asymptotic qualification condition the Lipschitzian-type global error bound property is equivalent to the Abadie qualification condition; in particular, the Lipschitzian-type global error bound is satisfied under the Slater condition. Second, without regularity conditions, the Hölderian global error bound with an explicit exponent is investigated.

Least quantile regression via modern optimization

We address the Least Quantile of Squares (LQS) (and in particular the Least Median of Squares) regression problem using modern optimization methods. We propose a Mixed Integer Optimization (MIO) formulation of the LQS problem which allows us to find a provably global optimal solution for the LQS problem. Our MIO framework has the appealing characteristic that if we terminate the algorithm early, we obtain a solution with a guarantee on its sub-optimality. We also propose continuous optimization methods based on first-order subdifferential methods, sequential linear optimization and hybrid combinations of them to obtain near optimal solutions to the LQS problem. The MIO algorithm is found to benefit significantly from high quality solutions delivered by our continuous optimization based methods. We further show that the MIO approach leads to (a) an optimal solution for any dataset, where the data-points $(y_i,\mathbf{x}_i)$'s are not necessarily in general position, (b) a simple proof of the breakdown point of the LQS objective value that holds for any dataset and (c) an extension to situations where there are polyhedral constraints on the regression coefficient vector. We report computational results with both synthetic and real-world datasets showing that the MIO algorithm with warm starts from the continuous optimization methods solve small ($n=100$) and medium ($n=500$) size problems to provable optimality in under two hours, and outperform all publicly available methods for large-scale ($n={}$10,000) LQS problems.

Approximate solution algorithm for multi-parametric non-convex programming problems with polyhedral constraints

In this paper, we developed a novel algorithmic approach for thesolution of multi-parametric non-convex programming problems withcontinuous decision variables. The basic idea of the proposedapproach is based on successive convex relaxation of each non-convexterms and sensitivity analysis theory. The proposed algorithm isimplemented using MATLAB software package and numericalexamples are presented to illustrate the effectiveness andapplicability of the proposed method on multi-parametric non-convexprogramming problems with polyhedral constraints.