Cutting-plane Research Articles

Query-efficient label-only attacks against black-box machine learning models

Recent studies have shown that machine learning algorithms are susceptible to imperceptible perturbations. These studies focus on the laboratory environment, where the attacker has knowledge about internal information of the victim model or feedbacks such as class probabilities. There is still a gap between theory and physical world, the risk of adversarial attacks under the more extreme and realistic condition needs to be figured out. Here we propose a knowledge restricted black-box attack model where the attacker can only get the final predict label. In the meantime, we model the attacker as a resource-restricted one, such as query-limited. The limitations of knowledge level and resources make previous work unable to be directly applied. For this problem, the current state-of-the-art method is boundary attack, however it requires large a number of queries. In this paper, we make several contributions to investigate the vulnerability of machine learning models in more realistic scenarios. First, we reconstruct the optimization problem, measure the quality of the sample points by L2 distance. Second, we provide a more effective algorithm, using cutting plane method and local optimization. Third, we propose two effective dynamic defense strategy, which is easy to implement. At last, we conduct an experimental evaluation on MNIST, Fashion-MNIST and malware detection datasets. The results show that (1) compared with state-of-the-art method, our cutting plane method reduces the number of queries while ensuring the attack efficiency; (2) Dynamic defense strategy is effective against label-only adversarial attacks, the rate of attack success dropped from nearly 100% to 23%, with a considerable classification accuracy; (3) Improved defense strategy guarantees the effectiveness of defense and improves the stability of the whole model.

On polyhedral and second-order cone decompositions of semidefinite optimization problems

We study a cutting-plane method for semidefinite optimization problems, and supply a proof of the method’s convergence, under a boundedness assumption. By relating the method’s rate of convergence to an initial outer approximation’s diameter, we argue the method performs well when initialized with a second-order cone approximation, instead of a linear approximation. We invoke the method to provide bound gaps of 0.5–6.5% for sparse PCA problems with 1000s of covariates, and solve nuclear norm problems over 500 × 500 matrices.

WOOD COLORIMETRY FOR THE CHARACTERIZATION OF AMAZONIAN TREE SPECIES: A SUBSIDY FOR A MORE EFFICIENT CLASSIFICATION

ABSTRACT The wood of different species may present macroscopic similarities, which hinders a reliable differentiation between the specie. This can be a problem, especially in the tropics, given the high species diversity in this region. To overcome this problem of wood identification, colorimetric criteria can be applied, aiming at a better and more objective classification, as opposed to the traditional anatomic criteria. Therefore, the aim of this research was to analyze the variability of the wood color considering the cutting planes (tangential and radial) and to determine its correlation with the wood basic density of ten Amazonian tree species. The following species were studied: Apuleia leiocarpa (amarelao), Hymenolobium modestum (angelim-pedra), Anadenanthera columbrina (angico), Dipteryx odorata (cumaru), Pouteria pachycarpa (goiabao), Mezilaurus itauba (itauba), Hymenaea courbaril (jatoba), Simarouba amara (marupa), Astronium leicontei (muiracatiara) and, Bowdichia nitida (sucupira). The CIEL*a*b* system was used to determine the colorimetric parameters (L*, a*, b*, C*, and h*), which were obtained through a portable colorimeter. The species were classified in six color specific groups: light yellow, olive brown, yellowish olive, olive light, grayish white and reddish brown. We observed statistical effect of the cut planes only for luminosity, with higher mean for radial face. The correlation analyses between color and basic density showed that the denser woods are darker and have more red pigment (a*). There is negative correlation between luminosity (L*) and wood basic density (-0.78). The colorimetric parameter a* presented positive correlation with the basic density (r = 0.80).

Mini-batch cutting plane method for regularized risk minimization

Although concern has been recently expressed with regard to the solution to the non-convex problem, convex optimization is still important in machine learning, especially when the situation requires an interpretable model. Solution to the convex problem is a global minimum, and the final model can be explained mathematically. Typically, the convex problem is re-casted as a regularized risk minimization problem to prevent overfitting. The cutting plane method (CPM) is one of the best solvers for the convex problem, irrespective of whether the objective function is differentiable or not. However, CPM and its variants fail to adequately address large-scale dataintensive cases because these algorithms access the entire dataset in each iteration, which substantially increases the computational burden and memory cost. To alleviate this problem, we propose a novel algorithm named the mini-batch cutting plane method (MBCPM), which iterates with estimated cutting planes calculated on a small batch of sampled data and is capable of handling large-scale problems. Furthermore, the proposed MBCPM adopts a “sink” operation that detects and adjusts noisy estimations to guarantee convergence. Numerical experiments on extensive real-world datasets demonstrate the effectiveness of MBCPM, which is superior to the bundle methods for regularized risk minimization as well as popular stochastic gradient descent methods in terms of convergence speed.

Computationally effective spot‐weld fatigue life estimation for vehicle components

AbstractTo further speed up the frequency‐domain fatigue life estimation procedure for vehicle components with large numbers of spot‐welds, several measures are proposed. A two‐scale FE‐approach based on a pre‐processed library for various spot‐weld‐ / sheet‐geometries, applying the cutting plane method, and a priori stress estimations reduce the necessary CPU‐time considerably and allow for an efficient consideration of different load histories.

Development cutting plane method to find optimal integer solution for Fuzzy linear programming problems

Development cutting plane method to find optimal integer solution for Fuzzy linear programming problems

Promoting zero-emissions vehicles using robust multi-period tradable credit scheme

This study designs a robust multi-period tradable credit scheme (TCS) to incentivize travelers to shift from internal combustion engine vehicles (ICEVs) to zero-emissions vehicles (ZEVs) over a long-term planning horizon to reduce vehicular emissions. The need for robust design arises because of uncertainty in forecasting travel demand over a planning horizon in the order of several years. The robust multi-period TCS design is formulated as a bi-level model. In the upper level, the central authority (CA) determines the TCS parameters (credit allocation and charging schemes) by vehicle type to minimize the worst-case vehicular emissions rate, i.e. the maximum vehicular emissions rate under the possible travel demand scenarios. The upper-level model is a mixed-integer nonlinear program. In the lower level, travelers minimize their generalized travel costs under the TCS parameters obtained in the upper level. These parameters are used to determine the vehicle type choice, between ICEVs and ZEVs, using a binomial logit function, and influence route selection based on the difference in credits charged on links for these two vehicle types. The lower-level model is a mathematical program with equilibrium constraints. The bi-level model is solved using a cutting plane method. Numerical experiments illustrate that the incentive to shift to ZEVs is fostered by allocating more credits and charging fewer credits to ZEV travelers compared to ICEV travelers. Further, the proposed TCS design reduces volatility in the realized vehicular emissions rates under different travel demand scenarios compared to a TCS design that does not consider demand uncertainty.

Large Margin Partial Label Machine.

Partial label learning (PLL) is a multi-class weakly supervised learning problem where each training instance is associated with a set of candidate labels but only one label is the ground truth. The main challenge of PLL is how to deal with the label ambiguities. Among various disambiguation techniques, large margin (LM)-based algorithms attract much attention due to their powerful discriminative performance. However, existing LM-based algorithms either neglect some potential candidate labels in constructing the margin or introduce auxiliary estimation of class capacities which is generally inaccurate. As a result, their generalization performances are deteriorated. To address the above-mentioned drawbacks, motivated by the optimistic superset loss, we propose an LM Partial LAbel machiNE (LM-PLANE) by extending multi-class support vector machines (SVM) to PLL. Compared with existing LM-based disambiguation algorithms, LM-PLANE considers the margin of all potential candidate labels without auxiliary estimation of class capacities. Furthermore, an efficient cutting plane (CP) method is developed to train LM-PLANE in the dual space. Theoretical insights into the effectiveness and convergence of our CP method are also presented. Extensive experiments on various PLL tasks demonstrate the superiority of LM-PLANE over existing LM based and other representative PLL algorithms in terms of classification accuracy.

An adjustable robust optimization approach for periodic timetabling

In this paper, we consider the Robust Periodic Timetabling Problem (RPTP), the problem of designing a periodic timetable that can easily be adjusted in case of small periodic disturbances. We develop a solution method for a parametrized class of uncertainty regions. This class relates closely to uncertainty regions known in the robust optimization literature, and naturally defines a metric for the robustness of the timetable. The proposed solution method combines a linear decision rule with well-known reformulation techniques and cutting-plane methods. We show that the RPTP can be solved for practical-sized instances by applying the solution method to practical cases of Netherlands Railways (NS). In particular, we show that the trade-off between the efficiency and robustness of a timetable can be analyzed using our solution method.

Cutting planes for families implying Frankl’s conjecture

We find previously unknown families which imply Frankl’s conjecture using an algorithmic framework. The conjecture states that for any non-empty union-closed (or Frankl) family there exists an element in at least half of the sets. Poonen’s Theorem characterizes the existence of weights which determine whether a given Frankl family implies the conjecture for all Frankl families which contain it. A Frankl family is Non–Frankl-Complete (Non–FC), if it does not imply the conjecture in its elements for some Frankl family that contains it. We design a cutting-plane method that computes the explicit weights which imply the existence conditions of Poonen’s Theorem. This method allows us to find a counterexample to a ten-year-old conjecture by R. Morris about the structure of generators for Non–FC-families.

Efficient frequency-domain based fatigue life estimation of spot welds in vehicle components

The estimation of the damage and fatigue life of spot welds in vehicle components with hundreds or thousands thereof within reasonable computation time is a challenge even for recent computers in automotive industry. Hence, an advanced procedure in the frequency-domain is proposed. Whereas the general frequency-domain approach is well-established, it nevertheless becomes efficient for a large number of spot welds only with a package of measures. Among others, a two-scale FE-method using a pre-processed data base for various spot weld/sheet geometries and specified basic load cases, application of the cutting-plane method, and a priori stress estimations to reduce the number of spot welds to be checked completely are suggested. By the example of a tank of a cooling medium in a vehicle it is shown that the proposed algorithm may reduce the computation time as compared to time-domain calculations substantially.

Portfolio optimization with relaxation of stochastic second order dominance constraints via conditional value at risk

A portfolio optimization model with relaxed second order stochastic dominance (SSD) constraints is presented. The proposed model uses Conditional Value at Risk (CVaR) constraints at probability level \begin{document}$ \beta\in(0,1) $\end{document} to relax SSD constraints. The relaxation is justified by theoretical convergence results based on sample average approximation (SAA) method when sample size \begin{document}$ N\to\infty $\end{document} and CVaR probability level \begin{document}$ \beta $\end{document} tends to 1. SAA method is used to reduce infinite number of inequalities of SSD constraints to finite ones and also to calculate the expectation value. The proposed relaxation on the SSD constraints in portfolio optimization problem is achieved when the probability level \begin{document}$ \beta $\end{document} of CVaR takes value less than but close to 1, and the model can then be solved by cutting plane method. The performance and characteristics of the portfolios constructed by solving the proposed model are tested empirically on three sets of market data, and the experimental results are analyzed and discussed. Furthermore, it is shown that with appropriate choices of CVaR probability level \begin{document}$ \beta $\end{document} , the constructed portfolios are sparse and outperform the portfolios constructed by solving portfolio optimization problems with SSD constraints, with either index portfolios or mean-variance (MV) portfolios as benchmarks.

A two-stage robust model for a reliable p-center facility location problem

We propose a two-stage robust model for reliable facility location when some facilities can be disrupted, for instance by a natural disaster. A reliable network is designed in a “proactive” planning phase, and when a facility is disrupted, its original clients can be reallocated to another available facility in a “reactive” phase. When demand and cost are uncertain, the initial design is also robust against the realizations (scenarios) of these parameters, which will only be revealed post-disruption. Based on the p-center location model, which attempts to optimize the worst-case performance of the network, our model is concerned with the reliability for every client. Three solution methods have been implemented and tested to solve the model, namely, a linear reformulation, a Benders dual cutting plane method, and a column-and-constraint generation method. We present an extensive numerical study to compare the performance of these methods. We find that, depending on the size of the instance (as given by the number of client sites and scenarios), either the Benders dual cutting plane method or column-and-constraint generation performs best. The effectiveness of our model is also examined in comparison with alternative facility location models.

An oracle for the discrete-time integral quadratic constraint problem

In the theory of integral quadratic constraints (IQCs), the condition for robust performance is given in terms of a frequency-domain inequality. This inequality can be transformed into an equivalent finite-dimensional linear matrix inequality by the application of the Kalman–Yakubovich–Popov (KYP) lemma and the introduction of an auxiliary matrix variable. Thus, the IQC problem can be expressed as a semi-definite program (SDP) and solved using convex optimization tools such as general-purpose SDP solvers based on interior point methods. However, by introducing the auxiliary variable, the size of the IQC problem significantly increases, and the aforementioned solvers may fail due to the high computational cost of the SDP. To address this issue, specialized SDP solvers have been developed to efficiently solve the SDP by exploiting its structure. An alternative approach consists of using cutting plane methods to directly solve the frequency-domain inequality without resorting to the KYP lemma. Cutting plane methods require an oracle that confirms if a given candidate solution satisfies the frequency-domain inequality. If the candidate solution is not feasible, the oracle generates a hyperplane that separates the candidate solution from the set of feasible solutions. In this paper, the analytic center cutting plane method is applied to the discrete-time IQC problem. The oracle for the discrete-time IQC problem is based on the spectrum of an extended symplectic matrix pencil and does not require an invertibility assumption that holds in continuous-time but not necessarily in discrete-time. An illustrative example is provided that shows the efficacy of the proposed method and points out directions for future research.

The procedure for the complete updating of immersive sets in one cutting plane method

In order to improve the practical implementation of the previously proposed cutting plane method, we have developed a procedure for updating immersive polyhedral sets in it. These sets successively approximate the epigraphs of some auxiliary functions that are being built at each step of the method. Each of the approximating sets is obtained by cutting off the current iterative point from the previous set by the support hyperplane. In connection with this, additional linear inequalities that form approximating sets are accumulated with an increase in the number of steps. This accumulation of additional constraints leads to an increase in the complexity of solving linear programming problems, which are set up in the method for finding iterative points. The procedure for updating approximating sets developed and included in the method allows periodically discarding accumulating cutting planes, which greatly simplifies the solution of auxiliary problems. This procedure is based on the introduced criterion for estimating the quality of the approximation of epigraphs. At those iterations, where the quality of the approximation becomes good enough, the mentioned updates occur. The convergence of the cutting plane method with the updating procedure included in it is proved. The implementation of the method is discussed.

Location of a conservative hyperplane for cutting plane methods in disjoint bilinear programming

Although several classes of cutting plane methods for deterministically solving disjoint bilinear programming (DBLP) have been proposed, the frequently encountered computational issue regarding the generation of a suitable cut from a degenerate vertex in a pseudo-global minimizer (PGM) still remains. Among the approaches to dealing with degeneracy, the most recent one is to generate a conservative cut. Nevertheless, the computational performance of the corresponding distance-following algorithm for its location seems far from satisfactory. This paper proposes several approaches that can be utilized to efficiently locate a conservative hyperplane from a degenerate vertex in a PGM. Extensive experiments are conducted to evaluate their performance from the dimensionality as well as the degree of degeneracy. From the computational viewpoint, these new approaches can outperform the earlier developed distance-following algorithm, and thereby can be incorporated into cutting plane methods for solving DBLP.

A cutting-plane method to nonsmooth multiobjective optimization problems

The cutting-plane optimization methods rely on the idea that any subgradient of the objective function or the active/violated constraints defines a halfspace to be excluded from a set that contains an optimal solution: the localizing set. This algorithm converges towards a global minimum of any pseudoconvex subdifferentiable function. A naive extension for multiobjective optimization would be using simultaneously some subgradients of all objective functions for a given feasible point. However, as demonstrated in this paper, this approach can lead to a convergence towards non-optimal points. This paper introduces an optimization strategy for cutting-plane methods to cope with multiobjective problems without any scalarization procedure. The proposed strategy guarantees that its optimal solution is a Pareto Optimal solution of the original problem, which is also no worse than the starting point, and that any Pareto Optimal solution can be sampled. Moreover, the auxiliary problem is infeasible only if the original problem is also infeasible. The new strategy inherits the original theoretical guarantees of cutting planes methods and it can be applied to build other strategies.

Cutting plane method in fuzzy environment

Cutting plane method in fuzzy environment

An effective hybrid approach to the two-stage capacitated facility location problem

The two-stage capacitated facility location problem (TSCFLP) aims to simultaneously determine the locations of plants and depots with limited capacities and the product flows from plants to depots and then to single source customers minimizing the total facility opening and transportation costs. In this paper, based on a cut-and-solve strategy for tree searching, a hybrid approach combining cutting plane techniques, local branching and kernel search is proposed to optimally solve the TSCFLP. In each iteration of the approach, a three-stage cutting plane method is first applied to obtain a tight lower bound and a local branching method is adopted to partition the problem into two disjoint subproblems based on the corresponding lower bound solution. The subproblem with a relatively small solution space is then exactly solved and pruned with the help of a kernel search technique. To evaluate the efficiency and the effectiveness of the proposed approach, extensive experiments on benchmark and newly generated instances of TSCFLP, as well as instances of a single-source capacitated facility location problem that is a reduced TSCFLP, are conducted. The experimental results show that our proposed approach significantly outperforms existing methods in the literature.

Solving Reverse Convex Programming Problems Using a Generalized Cutting-Plane Method

It is well known that each convex function $$f:{\mathbb{R}}^n \longrightarrow {\mathbb{R}}$$ is supremally generated by affine functions. More precisely, each convex function $$f:{\mathbb{R}}^n \longrightarrow {\mathbb{R}}$$ is the upper envelope of its affine minorants. In this paper, we propose an algorithm for solving reverse convex programming problems by using such a representation together with a generalized cutting-plane method. Indeed, by applying this representation, we solve a sequence of problems with a smaller feasible set, in which the reverse convex constraint is replaced by a still reverse convex but polyhedral constraint. Moreover, we prove that the proposed algorithm converges, under suitable assumptions, to an optimal solution of the original problem. This algorithm is coded in MATLAB language and is evaluated by some numerical examples.