Dual Optimal Solutions Research Articles

MTKSVCR: A novel multi-task multi-class support vector machine with safe acceleration rule

Regularized multi-task learning (RMTL) has shown good performance in tackling multi-task binary problems. Although RMTL can be used to handle multi-class problems based on “one-versus-one” and “one-versus-rest” techniques, the information of the samples is not fully utilized and the class imbalance problem occurs. Motivated by the regularization technique in RMTL, we propose an original multi-task multi-class model termed MTKSVCR based on “one-versus-one-versus-rest” strategy to achieve better testing accuracy. Due to the utilization of the idea of RMTL, the related information included in multiple tasks is mined by setting different penalty parameters before task-common and task-specific regularization terms. However, the proposed MTKSVCR is time-consuming since it employs all samples in each optimization problem. Therefore, a multi-parameter safe acceleration rule termed SA is further presented to reduce the time consumption. It identifies and deletes most of the superfluous samples corresponding to 0 elements in the dual optimal solution before solving. Then, only a reduced dual problem is to be solved and the computational efficiency is improved accordingly. The biggest advantage of the proposed SA lies in safety. Namely, it derives an identical optimal solution to the primal problem without SA. In addition, our method remains effective when multiple parameters change simultaneously. Experiments on different artificial datasets and benchmark datasets verify the validity of the proposed methods.

Linear programming sensitivity measured by the optimal value worst-case analysis

This paper introduces the concept of a derivative of the optimal value function in linear programming (LP). Basically, it is the worst case optimal value of an interval LP problem when the nominal data are inflated to intervals according to given perturbation patterns. By definition, the derivative expresses how the optimal value can worsen when the data are subject to variations. In addition, it also gives a certain sensitivity measure or condition number of an LP problem. If the LP problem is nondegenerate, the derivatives are easy to calculate from the computed primal and dual optimal solutions. For degenerate problems, the computation is more difficult. We propose an upper bound and some kind of characterization, but there are many open problems remaining. We carried out numerical experiments with specific LP problems and with real LP data from Netlib repository. They show that the derivatives give a suitable sensitivity measure of LP problems. It remains an open problem how to efficiently and rigorously handle degenerate problems.

A single-stage optimization procedure for data envelopment analysis

In data envelopment analysis, a conventional procedure for testing efficiency of decision making units (DMUs) consists of two stages, each requiring solution of a linear program. The first stage identifies the input or output radial efficiency of a DMU, and the second stage maximizes the sum of component input and output slacks. A traditional alternative is the single-stage approximation of the two-stage procedure in which the objective functions of the first and second stages are combined, with the latter multiplied by a small positive constant epsilon. A known drawback of such approach is that very small values of epsilon may cause computational inaccuracies and large values do not allow a good approximation. In this paper, we develop a new single-stage linear programming approach that does not require the specification of a small constant epsilon and is completely equivalent to the conventional two-stage solution procedure. It produces exactly the same sets of primal and dual optimal solutions as the two separate stages of the two-stage approach. The new single-stage procedure is applicable to any convex polyhedral technology.

Solving multi-attribute decision making problems with incomplete weights: A dual approach

This paper proposes a method of ranking discrete alternatives when attribute weights are incompletely known. There are a variety of situations in which it is reasonable to consider incomplete attribute weights and several techniques have been developed to solve such multi-attribute decision making problems. Most frequently, a linear programming (LP) problem subject to a set of incomplete attribute weights is solved to identify dominance relations between alternatives. In this paper, we explore a dual problem to find a closed-form solution and determine the extreme points of a set of (strictly) ranked attribute weights. A simple investigation of the dual optimal solution often leads to a preferred alternative and permits to find the optimal attribute weights that can be applied to the primal, based on the primal-dual relationship. Furthermore, we extend the approach to several examples of incomplete attribute weights and to linear partial information expressed as linear inequalities that satisfy some predefined conditions. Finally, we present a case study to demonstrate how the dual approach can be used to establish dominance between alternatives, when preference orders are specified for a subset of alternatives.

Communication‐aware distributed rebalancing for cooperative car‐sharing service

AbstractThis study presents a cooperative car‐sharing system in which various service providers jointly operate a sharing service. A distributed algorithm is proposed for rebalancing control to reduce deadhead vehicles as much as possible under the constraints of dispatchable vehicles and parking slots at car stations. Each provider cooperatively searches for the optimal solution by exchanging the estimated values of the optimal dual solutions over the communication networks between providers. Event‐driven communication is considered in which information exchange with the neighboring providers is performed only when the amount of change of the estimated optimal dual solution exceeds a threshold. Numerical instances explain that the estimations of all service providers meet the optimal rebalancing solutions.

Duality and sensitivity analysis of multistage linear stochastic programs

In this paper we investigate the dual of a Multistage Stochastic Linear Program (MSLP). By writing Dynamic Programming equations for the dual, we can employ an SDDP type method, called Dual SDDP, which solves these Dynamic Programming equations. allows us to compute a sequence of nonincreasing deterministic upper bounds for the optimal value of the problem. Since the Relatively Complete Recourse (RCR) condition may fail to hold for the dual (even for simple problems), we design two variants of Dual SDDP, namely Dual SDDP with penalizations and Dual SDDP with feasibility cuts, that converge to the optimal value of the dual (and therefore primal when there is no duality gap) problem under mild assumptions. We also show that optimal dual solutions can be obtained using dual information from Primal SDDP (applied to the original primal MSLP) subproblems.As a byproduct of the developed methodology we study sensitivity of the optimal value of the problem as a function of the involved parameters. For the sensitivity analysis we provide formulas for the derivatives of the value function with respect to the parameters and illustrate their application on an inventory problem. Since these formulas involve optimal dual solutions, we need an algorithm that computes such solutions to use them, i.e., we need to solve the dual problem.Finally, as a proof of concept of the tools developed, we present the results of numerical experiments computing the sensitivity of the optimal value of an inventory problem as a function of parameters of the demand process and compare Primal and Dual SDDP on the inventory and a hydro-thermal planning problems.

Cooperative voltage management by demand resources and fair payoff allocation for distribution systems

This paper proposes a socially acceptable framework for the realization of efficient voltage management by cooperation among consumers in electric distribution systems where the frequent voltage variation problem is caused by large amounts of photovoltaic (PV) generation systems. In the proposed framework, the profit maximization problem of the coalition under the constraints to suppress the voltage variations caused by PVs is formulated as a linear programming problem, and the payoff allocation among consumers is provided using a solution concept from cooperative game theory called the core. The proposed method efficiently computes the payoff allocation in the nonempty core to accomplish the cooperative voltage management system. Unlike most related works, this paper employs cooperative game theory to explicitly address incentives for cooperation with the consumers. Although the computation for finding an allocation in the core generally requires a significant amount of time, this paper demonstrates that the allocation can be efficiently calculated by using the dual optimal solution of a formulated linear programming problem. The proposed method is validated by computational experiments on a large-scale distribution system model in which consumers with demand resources (including PVs and controllable loads) are connected.

Hybrid NOMA-FDMA Assisted Dual Computation Offloading: A Latency Minimization Approach

Edge computing has been considered as a promising solution for enabling computation-intensive yet latency-sensitive applications at resource-constrained wireless devices (WDs). In this paper, exploiting the advanced small-cell dual connectivity (DC), we investigate a paradigm of dual computation offloading in which a WD can simultaneously offload partial workloads to a cloudlet-server co-located at the macro base station (MBS) and an edge-server (ES) co-located at a small-cell based station (SBS). To facilitate the multi-user dual computation offloading, we exploit a hybrid model of non-orthogonal multiple access (NOMA) and frequency division multiple access (FDMA). Specifically, due to the SBSs’ limited channel resources, we consider that the WDs form different NOMA-groups for offloading their respective workloads to different SBSs, which improves the spectrum efficiency. Meanwhile, all WDs use FDMA for offloading their workloads to the MBS, which avoids the WDs’ co-channel interference. We formulate a joint optimization of the WDs’ partial offloading decisions, their FDMA transmission to the MBS, different NOMA-groups’ transmission to the SBSs, as well as the computing-rate allocation of the ESs and the cloudlet-server, with the objective of minimizing the overall latency for completing all WDs’ workloads. Despite the strict non-convexity of the joint optimization problem, we propose a layered yet cell-based distributed algorithm for obtaining the optimal dual offloading solution. Based on the optimal dual offloading solution, we further investigate how to properly group WDs into different NOMA-groups for offloading workloads to the corresponding SBSs, and propose a cross-entropy based learning algorithm for finding the optimal NOMA grouping scheme. Numerical results are finally provided to validate the effectiveness and efficiency of our proposed algorithms.

Limit laws for empirical optimal solutions in random linear programs

We consider a general linear program in standard form whose right-hand side constraint vector is subject to random perturbations. For the corresponding random linear program, we characterize under general assumptions the random fluctuations of the empirical optimal solutions around their population quantities after standardization by a distributional limit theorem. Our approach is geometric in nature and further relies on duality and the collection of dual feasible basic solutions. The limiting random variables are driven by the amount of degeneracy inherent in linear programming. In particular, if the corresponding dual linear program is degenerate the asymptotic limit law might not be unique and is determined from the way the empirical optimal solution is chosen. Furthermore, we include consistency and convergence rates of the Hausdorff distance between the empirical and the true optimality sets as well as a limit law for the empirical optimal value involving the set of all dual optimal basic solutions. Our analysis is motivated from statistical optimal transport that is of particular interest here and distributional limit laws for empirical optimal transport plans follow by a simple application of our general theory. The corresponding limit distribution is usually non-Gaussian which stands in strong contrast to recent finding for empirical entropy regularized optimal transport solutions.

A Primal–Dual Learning Algorithm for Personalized Dynamic Pricing with an Inventory Constraint

We consider the problem of a firm seeking to use personalized pricing to sell an exogenously given stock of a product over a finite selling horizon to different consumer types. We assume that the type of an arriving consumer can be observed, but the demand function associated with each type is initially unknown. The firm sets personalized prices dynamically for each type and attempts to maximize the revenue over the season. We provide a learning algorithm that is near optimal when the demand and capacity scale in proportion. The algorithm utilizes the primal–dual formulation of the problem and learns the dual optimal solution explicitly. It allows the algorithm to overcome the curse of dimensionality (the rate of regret is independent of the number of types) and sheds light on novel algorithmic designs for learning problems with resource constraints.

Convergence rates of RLT and Lasserre-type hierarchies for the generalized moment problem over the simplex and the sphere

We consider the generalized moment problem (GMP) over the simplex and the sphere. This is a rich setting and it contains NP-hard problems as special cases, like constructing optimal cubature schemes and rational optimization. Using the reformulation-linearization technique (RLT) and Lasserre-type hierarchies, relaxations of the problem are introduced and analyzed. For our analysis we assume throughout the existence of a dual optimal solution as well as strong duality. For the GMP over the simplex we prove a convergence rate of O(1/r) for a linear programming, RLT-type hierarchy, where r is the level of the hierarchy, using a quantitative version of Pólya’s Positivstellensatz. As an extension of a recent result by Fang and Fawzi (Math Program, 2020. https://doi.org/10.1007/s10107-020-01537-7) we prove the Lasserre hierarchy of the GMP (Lasserre in Math Program 112(1):65–92, 2008. https://doi.org/10.1007/s10107-006-0085-1) over the sphere has a convergence rate of O(1/r^2). Moreover, we show the introduced linear RLT-relaxation is a generalization of a hierarchy for minimizing forms of degree d over the simplex, introduced by De Klerk et al. (J Theor Comput Sci 361(2–3):210–225, 2006).

Stabilized Column Generation Via the Dynamic Separation of Aggregated Rows

Column generation (CG) algorithms are well known to suffer from convergence issues due, mainly, to the degenerate structure of their master problem and the instability associated with the dual variables involved in the process. In the literature, several strategies have been proposed to overcome this issue. These techniques rely either on the modification of the standard CG algorithm or on some prior information about the set of dual optimal solutions. In this paper, we propose a new stabilization framework, which relies on the dynamic generation of aggregated rows from the CG master problem. To evaluate the performance of our method and its flexibility, we consider instances of three different problems, namely, vehicle routing with time windows (VRPTW), bin packing with conflicts (BPPC), and multiperson pose estimation (MPPEP). When solving the VRPTW, the proposed stabilized CG method yields significant improvements in terms of CPU time and number of iterations with respect to a standard CG algorithm. Huge reductions in CPU time are also achieved when solving the BPPC and the MPPEP. For the latter, our method has shown to be competitive when compared with a tailored method. Summary of Contribution: Column generation (CG) algorithms are among the most important and studied solution methods in operations research. CG algorithms are suitable to cope with large-scale problems arising from several real-life applications. The present paper proposes a generic stabilization framework to address two of the main issues found in a CG method: degeneracy in the master problem and massive instability of the dual variables. The newly devised method, called dynamic separation of aggregated rows (dyn-SAR), relies on an extended master problem that contains redundant constraints obtained by aggregating constraints from the original master problem formulation. This new formulation is solved in a column/row generation fashion. The efficacy of the proposed method is tested through an extensive experimental campaign, where we solve three different problems that differ considerably in terms of their constraints and objective function. Despite being a generic framework, dyn-SAR requires the embedded CG algorithm to be tailored to the application at hand.

The strict complementarity in linear fractional optimization

As an important duality result in linear optimization, the Goldman–Tucker theorem establishes strict complementarity between a pair of primal and dual linear programs. Our study extends this result into the framework of linear fractional optimization. Associated with a linear fractional program, a dual program can be defined as the dual of the equivalent linear program obtained from applying the Charnes–Cooper transformation to the given program. Based on this definition, we propose new criteria for primal and dual optimality by showing that the primal and dual optimal sets can be equivalently modeled as the optimal sets of a pair of primal and dual linear programs. Then, we define the concept of strict complementarity and establish the existence of at least one, called strict complementary, pair of primal and dual optimal solutions such that in every pair of comple mentary variables, exactly one variable is positive and the other is zero. We geometrically interpret the strict complementarity in terms of the relative interiors of two sets that represent the primal and dual optimal setsin higher dimensions. Finally, using this interpretation, we develop two approaches for finding a strict complementary solution in linear fractional optimization. We illustrate our results with two numerical examples.

Improving ADMMs for solving doubly nonnegative programs through dual factorization

Alternating direction methods of multipliers (ADMMs) are popular approaches to handle large scale semidefinite programs that gained attention during the past decade. In this paper, we focus on solving doubly nonnegative programs (DNN), which are semidefinite programs where the elements of the matrix variable are constrained to be nonnegative. Starting from two algorithms already proposed in the literature on conic programming, we introduce two new ADMMs by employing a factorization of the dual variable. It is well known that first order methods are not suitable to compute high precision optimal solutions, however an optimal solution of moderate precision often suffices to get high quality lower bounds on the primal optimal objective function value. We present methods to obtain such bounds by either perturbing the dual objective function value or by constructing a dual feasible solution from a dual approximate optimal solution. Both procedures can be used as a post-processing phase in our ADMMs. Numerical results for DNNs that are relaxations of the stable set problem are presented. They show the impact of using the factorization of the dual variable in order to improve the progress towards the optimal solution within an iteration of the ADMM. This decreases the number of iterations as well as the CPU time to solve the DNN to a given precision. The experiments also demonstrate that within a computationally cheap post-processing, we can compute bounds that are close to the optimal value even if the DNN was solved to moderate precision only. This makes ADMMs applicable also within a branch-and-bound algorithm.

On the existence of a short pivoting sequence for a linear program

We show that given a feasible primal–dual pair of linear programs in canonical form, there exists a sequence of pivots, whose length is bounded by the minimum dimension of the constraint matrix, leading from the origin to the optimum. The sequence of pivots give a sequence of square and nonsingular submatrices of the constraint matrix. Solving two linear equations involving such a submatrix give primal–dual optimal solutions to the corresponding linear program in canonical form.

Relationship of two formulations for shortest bibranchings

The shortest bibranching problem is a common generalization of the minimum-weight edge cover problem in bipartite graphs and the minimum-weight arborescence problem in directed graphs. For the shortest bibranching problem, an efficient primal-dual algorithm is given by Keijsper and Pendavingh (J Comb Theory Ser B 73:130–145, 1998), and the tractability of the problem is ascribed to total dual integrality in a linear programming formulation by Schrijver (Ann Discret Math 16:261–280, 1982). Another view on the tractability of this problem is afforded by a valuated matroid intersection formulation by Takazawa (Jpn J Ind Appl Math 29:561–573, 2012). In the present paper, we discuss the relationship between these two formulations for the shortest bibranching problem. We first demonstrate that the valuated matroid intersection formulation can be derived from the linear programming formulation through the Benders decomposition, where integrality is preserved in the decomposition process and the resulting convex programming is endowed with discrete convexity. We then show how a pair of primal and dual optimal solutions of one formulation is constructed from that of the other formulation, thereby providing a connection between polyhedral combinatorics and discrete convex analysis.

Secure Millimeter Wave Cloud Radio Access Networks Relying on Microwave Multicast Fronthaul

In this paper, we investigate the downlink secure beamforming (BF) design problem of cloud radio access networks (C-RANs) relying on multicast fronthaul, where millimeter-wave and microwave carriers are used for the access links and fronthaul links, respectively. The base stations (BSs) jointly serve users through cooperating hybrid analog/digital BF. We first develop an analog BF for cooperating BSs. On this basis, we formulate a secrecy rate maximization (SRM) problem subject both to a realistic limited fronthaul capacity and to the total BS transmit power constraint. Due to the intractability of the non-convex problem formulated, advanced convex approximated techniques, constrained concave convex procedures and semi-definite programming (SDP) relaxation are applied to transform it into a convex one. Subsequently, an iterative algorithm of jointly optimizing multicast BF, cooperative digital BF and the artificial noise (AN) covariance is proposed. Next, we construct the solution of the original problem by exploiting both the primal and the dual optimal solution of the SDP-relaxed problem. Furthermore, a per-BS transmit power constraint is considered, necessitating the reformulation of the SRM problem, which can be solved by an efficient iterative algorithm. We then eliminate the idealized simplifying assumption of having perfect channel state information (CSI) for the eavesdropper links and invoke realistic imperfect CSI. Furthermore, a worst-case SRM problem is investigated. Finally, by combining the so-called $\mathcal {S}$ -Procedure and convex approximated techniques, we design an efficient iterative algorithm to solve it. Simulation results are presented to evaluate the secrecy rate and demonstrate the effectiveness of the proposed algorithms.

Tracking-ADMM for distributed constraint-coupled optimization

We consider constraint-coupled optimization problems in which agents of a network aim to cooperatively minimize the sum of local objective functions subject to individual constraints and a common linear coupling constraint. We propose a novel optimization algorithm that embeds a dynamic average consensus protocol in the parallel Alternating Direction Method of Multipliers (ADMM) to design a fully distributed scheme for the considered set-up. The dynamic average mechanism allows agents to track the time-varying coupling constraint violation (at the current solution estimates). The tracked version of the constraint violation is then used to update local dual variables in a consensus-based scheme mimicking a parallel ADMM step. Under convexity, we prove that all limit points of the agents’ primal solution estimates form an optimal solution of the constraint-coupled (primal) problem. The result is proved by means of a Lyapunov-based analysis simultaneously showing consensus of the dual estimates to a dual optimal solution, convergence of the tracking scheme and asymptotic optimality of primal iterates. A numerical study on optimal charging schedule of plug-in electric vehicles corroborates the theoretical results.

Dual-based methods for solving infinite-horizon nonstationary deterministic dynamic programs

We develop novel dual-ascent and primal-dual methods to solve infinite-horizon nonstationary deterministic dynamic programs. These methods are finitely implementable and converge in value to optimality. Moreover, the dual-ascent method produces a sequence of improving dual solutions that pointwise converge to an optimal dual solution, while the primal-dual algorithm provides a sequence of primal basic feasible solutions with value error bounds from optimality that converge to zero. Our dual-based methods work on a more general class of infinite network flow problems that include the shortest-path formulation of dynamic programs as a special case. To our knowledge, these are the first dual-based methods proposed in the literature to solve infinite-horizon nonstationary deterministic dynamic programs.

High-dimensional model recovery from random sketched data by exploring intrinsic sparsity

Learning from large-scale and high-dimensional data still remains a computationally challenging problem, though it has received increasing interest recently. To address this issue, randomized reduction methods have been developed by either reducing the dimensionality or reducing the number of training instances to obtain a small sketch of the original data. In this paper, we focus on recovering a high-dimensional classification/regression model from random sketched data. We propose to exploit the intrinsic sparsity of optimal solutions and develop novel methods by increasing the regularization parameter before the sparse regularizer. In particular, (i) for high-dimensional classification problems, we leverage randomized reduction methods to reduce the dimensionality of data and solve a dual formulation on the random sketched data with an introduced sparse regularizer on the dual solution; (ii) for high-dimensional sparse least-squares regression problems, we employ randomized reduction methods to reduce the scale of data and solve a formulation on the random sketched data with an increased regularization parameter before the sparse regularizer. For both classes of problems, by exploiting the intrinsic sparsity of the optimal dual solution or the optimal primal solution we provide formal theoretical guarantee on the recovery error of learned models in comparison with the optimal models that are learned from the original data. Compared with previous studies on randomized reduction for machine learning, the present work enjoy several advantages: (i) the proposed formulations enjoys intuitive geometric explanations; (ii) the theoretical guarantee does not rely on any stringent assumptions about the original data (e.g., low-rankness of the data matrix or the data are linearly separable); (iii) the theory covers both smooth and non-smooth loss functions for classification; (iv) the analysis is applicable to a broad class of randomized reduction methods as long as the reduction matrices admit the Johnson–Lindenstrauss type of lemma. We also present empirical studies to support the proposed methods and the presented theory.