Min-max Optimization Problem Research Articles

Robust Optimal Classification Trees against Adversarial Examples

Decision trees are a popular choice of explainable model, but just like neural networks, they suffer from adversarial examples. Existing algorithms for fitting decision trees robust against adversarial examples are greedy heuristics and lack approximation guarantees. In this paper we propose ROCT, a collection of methods to train decision trees that are optimally robust against user-specified attack models. We show that the min-max optimization problem that arises in adversarial learning can be solved using a single minimization formulation for decision trees with 0-1 loss. We propose such formulations in Mixed-Integer Linear Programming and Maximum Satisfiability, which widely available solvers can optimize. We also present a method that determines the upper bound on adversarial accuracy for any model using bipartite matching. Our experimental results demonstrate that the existing heuristics achieve close to optimal scores while ROCT achieves state-of-the-art scores.

Training Robust Deep Models for Time-Series Domain: Novel Algorithms and Theoretical Analysis

Despite the success of deep neural networks (DNNs) for real-world applications over time-series data such as mobile health, little is known about how to train robust DNNs for time-series domain due to its unique characteristics compared to images and text data. In this paper, we fill this gap by proposing a novel algorithmic framework referred as RObust Training for Time-Series (RO-TS) to create robust deep models for time-series classification tasks. Specifically, we formulate a min-max optimization problem over the model parameters by explicitly reasoning about the robustness criteria in terms of additive perturbations to time-series inputs measured by the global alignment kernel (GAK) based distance. We also show the generality and advantages of our formulation using the summation structure over time-series alignments by relating both GAK and dynamic time warping (DTW). This problem is an instance of a family of compositional min-max optimization problems, which are challenging and open with unclear theoretical guarantee. We propose a principled stochastic compositional alternating gradient descent ascent (SCAGDA) algorithm for this family of optimization problems. Unlike traditional methods for time-series that require approximate computation of distance measures, SCAGDA approximates the GAK based distance on-the-fly using a moving average approach. We theoretically analyze the convergence rate of SCAGDA and provide strong theoretical support for the estimation of GAK based distance. Our experiments on real-world benchmarks demonstrate that RO-TS creates more robust deep models when compared to adversarial training using prior methods that rely on data augmentation or new definitions of loss functions. We also demonstrate the importance of GAK for time-series data over the Euclidean distance.

Learning Universal Adversarial Perturbation by Adversarial Example

Deep learning models have shown to be susceptible to universal adversarial perturbation (UAP), which has aroused wide concerns in the community. Compared with the conventional adversarial attacks that generate adversarial samples at the instance level, UAP can fool the target model for different instances with only a single perturbation, enabling us to evaluate the robustness of the model from a more effective and accurate perspective. The existing universal attack methods fail to exploit the differences and connections between the instance and universal levels to produce dominant perturbations. To address this challenge, we propose a new universal attack method that unifies instance-specific and universal attacks from a feature perspective to generate a more dominant UAP. Specifically, we reformulate the UAP generation task as a minimax optimization problem and then utilize the instance-specific attack method to solve the minimization problem thereby obtaining better training data for generating UAP. At the same time, we also introduce a consistency regularizer to explore the relationship between training data, thus further improving the dominance of the generated UAP. Furthermore, our method is generic with no additional assumptions about the training data and hence can be applied to both data-dependent (supervised) and data-independent (unsupervised) manners. Extensive experiments demonstrate that the proposed method improves the performance by a significant margin over the existing methods in both data-dependent and data-independent settings. Code is available at https://github.com/lisenxd/AT-UAP.

A Generative Model for Texture Synthesis based on Optimal Transport Between Feature Distributions

We propose GOTEX, a general framework for texture synthesis by optimization that constrains the statistical distribution of local features. While our model encompasses several existing texture models, we focus on the case where the comparison between feature distributions relies on optimal transport distances. We show that the semi-dual formulation of optimal transport allows to control the distribution of various possible features, even if these features live in a high-dimensional space. We then study the resulting minimax optimization problem, which corresponds to a Wasserstein generative model, for which the inner concave maximization problem can be solved with standard stochastic gradient methods. The alternate optimization algorithm is shown to be versatile in terms of applications, features and architecture; in particular, it allows to produce high-quality synthesized textures with different sets of features. We analyze the results obtained by constraining the distribution of patches or the distribution of responses to a pre-learned VGG neural network. We show that the patch representation can retrieve the desired textural aspect in a more precise manner. We also provide a detailed comparison with state-of-the-art texture synthesis methods. The GOTEX model based on patch features is also adapted to texture inpainting and texture interpolation. Finally, we show how to use our framework to learn a feed-forward neural network that can synthesize on-the-fly new textures of arbitrary size in a very fast manner. Experimental results and comparisons with the mainstream methods from the literature illustrate the relevance of the generative models learned with GOTEX.

An H∞ approach to data‐driven fault estimation, and isolation for Hammerstein‐Wiener systems

AbstractThis article considers the fault estimation problem in a data‐driven setup for a class of nonlinear systems called Hammerstein‐Wiener. The fault estimation problem is converted to a finite‐horizon minimax optimization problem. To estimate faults, a subspace predictor is utilized to predict the output sequence. Our proposed method is robust against disturbances. The proposed method is briefly expressed as an algorithm that can be implemented by MATLAB software. The suggested algorithm is applied to a numerical MIMO example to verify its validity.

Optimal Design of Tuned Mass-Damper-Inerter for Structure with Uncertain-but-Bounded Parameter

In this study we focus on the H∞ optimization of a tuned mass damper inerter (TMDI), which is implemented on an harmonically forced structure of a single degree of freedom in the presence of stiffness uncertainty. Posed as a min-max optimization problem, its closed-form solutions are analytically derived via an algebraic approach that was newly developed in this work, and ready-to-use formulae of tuning parameters are provided herein for the optimal TMDI (referred to as the TMD). The accuracy of the derived solutions are examined by comparing them with the existing literature and with numerically solved solutions in both deterministic and uncertain scenarios. Our numerical investigation suggested that compared to the classic design, the proposed tuning strategy could effectively reduce the peak vibration amplitude of the host structure in the worst-case scenario. Moreover, its peak vibration amplitude decreases monotonically as the total amount of the tuned mass and inertance increases. Therefore, the incorporation of a grounded inerter in a traditional TMD could render the deteriorated performance of vibration control less important, thereby protecting the primary system against the detuning effect more effectively. Finally, the effectiveness of the proposed design under random excitation is also underlined.

An Optimal Transport Approach to Personalized Federated Learning

Federated learning is a distributed machine learning paradigm, which aims to train a model using the local data of many distributed clients. A key challenge in federated learning is that the data samples across the clients may not be identically distributed. To address this challenge, personalized federated learning with the goal of tailoring the learned model to the data distribution of every individual client has been proposed. In this paper, we focus on this problem and propose a novel personalized Federated Learning scheme based on Optimal Transport (FedOT) as a learning algorithm that learns the optimal transport maps for transferring data points to a common distribution as well as the prediction model under the applied transport map. To formulate the FedOT problem, we extend the standard optimal transport task between two probability distributions to multi-marginal optimal transport problems with the goal of transporting samples from multiple distributions to a common probability domain. We then leverage the results on multi-marginal optimal transport problems to formulate FedOT as a min-max optimization problem and analyze its generalization and optimization properties. We discuss the results of several numerical experiments to evaluate the performance of FedOT under heterogeneous data distributions in federated learning problems.

Unsupervised Learning Discriminative MIG Detectors in Nonhomogeneous Clutter

Principal component analysis (PCA) is a commonly used pattern analysis method that maps high-dimensional data into a lower-dimensional space maximizing the data variance, that results in the promotion of separability of data. Inspired by the principle of PCA, a novel type of learning discriminative matrix information geometry (MIG) detectors in the unsupervised scenario are developed, and applied to signal detection in nonhomogeneous environments. Hermitian positive-definite (HPD) matrices can be used to model the sample data, while the clutter covariance matrix is estimated by the geometric mean of a set of secondary HPD matrices. We define a projection that maps the HPD matrices in a high-dimensional manifold to a low-dimensional and more discriminative one to increase the degree of separation of HPD matrices by maximizing the data variance. Learning a mapping can be formulated as a two-step mini-max optimization problem in Riemannian manifolds, which can be solved by the Riemannian gradient descent algorithm. Three discriminative MIG detectors are illustrated with respect to different geometric measures, i.e., the Log-Euclidean metric, the Jensen–Bregman LogDet divergence and the symmetrized Kullback–Leibler divergence. Simulation results show that performance improvements of the novel MIG detectors can be achieved compared with the conventional detectors and their state-of-the-art counterparts within nonhomogeneous environments.

Stability of bimodal planar switched linear systems with both stable and unstable subsystems

We study dynamics of a bimodal planar linear switched system with a Hurwitz stable and an unstable subsystem. For given flee time from the unstable subsystem, the goal is to find corresponding dwell time in the Hurwitz stable subsystem so that the switched system is asymptotically stable. The dwell-flee relations obtained are in terms of certain smooth functions of the eigenvalues and (generalised) eigenvectors of the subsystem matrices. The results are extended to a special class of symmetric bilinear systems. The results are also extended to a multimodal planar linear switched system in which the switching is governed by an undirected star graph, where the internal node corresponds to a Hurwitz stable (unstable, respectively) subsystem and all the leaves correspond to unstable (Hurwitz stable, respectively) subsystems. For this multimodal system, dwell-flee relations are obtained as solutions of a minimax optimisation problem.

Design of maximum-stability PID controllers for LTI systems based on a stabilizing-set construction method

BackgroundThe PID algorithm has been widely used in control of chemical processes. A maximum-stability PID controller can provide superior stability robustness towards plant's variations and maximum exponential decay rate for disturbance rejection. However, design of maximum-stability PID controller is a min-max optimization problem and an effective method is still lack in the literature. MethodsBased on the characterization of stabilizing PID controller set, an efficient algorithm is developed to test if a plant is σ-stabilizable, where σ is abscissa or stability degree of the Hurwitz stable closed-loop characteristic polynomial. This algorithm is then used along with a bisection strategy to find a σ-interval [σε*,σε*+ε] which contains the maximum stability degree σ* for a specified ε, and the PID controller parameter set for achieving the stability degree σε*. Significant findingsThis paper has presented a systematic and efficient approach to design PID controllers with maximum degree of stability. The principal results include: (i) an improved theorem is presented for identifying stabilizing kp-intervals such that unnecessary computations are avoided; (ii) a simple yet effective method has been adopted to provide a non-conservative interval of σ which facilitates the bisectional branch-and-bound operation; (iii) the design procedure does not involve the actual construction of stabilizing PID controller sets thus renders its efficiency.

Topological synthesis of fluidic pressure-actuated robust compliant mechanisms

This paper presents a robust density-based topology optimization approach for synthesizing pressure-actuated compliant mechanisms. To ensure functionality under manufacturing inaccuracies, the robust or three-field formulation is employed, involving dilated, intermediate and eroded realizations of the design. Darcy’s law in conjunction with a conceptualized drainage term is used to model the pressure load as a function of the design vector. The consistent nodal loads are evaluated from the obtained pressure field using the standard finite element method. The objective and load sensitivities are obtained using the adjoint-variable approach. A multi-criteria objective involving both the stiffness and flexibility of the mechanism is employed in the robust formulation, and min–max optimization problems are solved to obtain pressure-actuated inverter, gripper, and contractor compliant mechanisms with different minimum feature sizes. Limitations of the linear elasticity assumptions while designing mechanisms are identified with high pressure loads. Challenges involved in designing finite deformable pressure-actuated compliant mechanisms are presented.

Wide-area damping power system control based on robust self-triggered model predictive control

Unavoidable random disturbances during the operation of large-scale interconnected power systems may undermine their stability. To prevent this problem, we propose a damping control method based on robust self-triggered model predictive control. First, we linearize the power system model and determine the controller installation location and feedback signal through geometric measurements. Subsequently, the model order is reduced by applying Schur method for balanced truncation. According to the assumed distribution and boundary range of random disturbance, the multi-cell robust predictive control minimax optimization problem described by the vertex is established. By deriving the upper bound of the maximum problem, the minimax problem is transformed into minimizing the upper bound problem to deal with the uncertain problem, which is transformed into linear matrix inequality for controller design. Finally, we introduce a self-triggered control strategy that avoids solving a constrained optimization problem at every time step. This strategy reduces the number of controller responses and resource requirements for sampling and communication, thereby improving computational efficiency. Selecting the two zone four-machine system as the test system, the simulation results demonstrate that the proposed control strategy can effectively eliminate the effect of random disturbances on the stable operation of power systems, improve the damping characteristics of power systems, and suppress low-frequency oscillations.

Generative Adversarial Training for Supervised and Semi-supervised Learning.

Neural networks have played critical roles in many research fields. The recently proposed adversarial training (AT) can improve the generalization ability of neural networks by adding intentional perturbations in the training process, but sometimes still fail to generate worst-case perturbations, thus resulting in limited improvement. Instead of designing a specific smoothness function and seeking an approximate solution used in existing AT methods, we propose a new training methodology, named Generative AT (GAT) in this article, for supervised and semi-supervised learning. The key idea of GAT is to formulate the learning task as a minimax game, in which the perturbation generator aims to yield the worst-case perturbations that maximize the deviation of output distribution, while the target classifier is to minimize the impact of this perturbation and prediction error. To solve this minimax optimization problem, a new adversarial loss function is constructed based on the cross-entropy measure. As a result, the smoothness and confidence of the model are both greatly improved. Moreover, we develop a trajectory-preserving-based alternating update strategy to enable the stable training of GAT. Numerous experiments conducted on benchmark datasets clearly demonstrate that the proposed GAT significantly outperforms the state-of-the-art AT methods in terms of supervised and semi-supervised learning tasks, especially when the number of labeled examples is rather small in semi-supervised learning.

On approximate optimality conditions for robust multi-objective convex optimization problems

In this paper, we are interested in the study of approximate optimality conditions for weakly ϵ-efficient solutions to robust multi-objective optimization problems ((RMOP) for short) in view of its associated minimax optimization problem (MMOP). To this end, we first establish the relationship between a weakly ϵ-efficient solution to the problem (RMOP) and an α-solution to the problem (MMOP), where and . Then, we explore the representation of the so-called β-normal set (where is a given parameter) to a closed convex set at some reference point by two methods. At last, by employing the α-subdifferential of the max-function and the obtained representation of the β-normal set, we establish an approximate necessary optimality condition for the problem (RMOP). Moreover, we also give an example to illustrate our results.

A novel privacy-aware model for nonparametric decentralized detection

In recent years, the increasing development of the Internet of Things (IoT) demands the enhancement of both network security and user privacy protection. In the decentralized IoT network, multiple sensors send local observations to a fusion center for data aggregation and authorized hypothesis detection. But at the same time, private information might be inferred illegally, which would cause privacy leakage. In this paper, a novel privacy-aware model named AL-UP is proposed for the nonparametric decentralized detection in the IoT network. It aims to design a local differential privacy and data projection based sanitization mechanism for sensors, to hide the sensitive information in raw observations and protect the data and inference privacy. Based on the adversarial learning framework and linear discriminant analysis, we propose a max-min optimization problem to design parameters of the sanitization mechanism and hypothesis detection rules. The problem is solved via the block coordinate descend method. Numerical results on various public datasets indicate that the proposed model achieves better utility-privacy trade-off than the state of the arts.

Secrecy‐oriented user association in ultra dense heterogeneous networks against strategically colluding adversaries

Network densification is recognized as the key technology to meet the ever growing demand of data traffic in next generation wireless networks. However the proliferation of small cell base stations (SCBSs) introduces vulnerabilities to information security, as they are prone to eavesdropping attacks. In this paper, it is studied how user association strategies can be specifically tailored to meet this security challenge in a ultra dense heterogeneous cellular network(UDHCN) with a group of colluding eavesdroppers. In particular, the situation is considered where eavesdroppers may have limited capabilities in intercepting and decoding data traffic. In this setting each eavesdropper has to make its own decision on eavesdropping targets, and they are able to take concerted actions to sabotage information security. To address this challenge, a zero sum game framework to reflect the conflicting interests of users and eavesdroppers is proposed and a user association scheme is devised that aims to maximize system sum secrecy rate against these adversaries whose actions intend to minimize sum secrecy rate. The case that all adversaries have limited eavesdropping capabilities is first considered, and it is shown that the corresponding zero sum game is indeed a bilinear game, and its Nash equilibrium solution can be readily approximated using a saddle point Frank Wolfe(SP-FW)algorithm. Then the framework is extended to the case where adversaries with a diverse configuration of eavesdropping capabilities exist. In this case, it is shown that the underlying minimax optimization problem is indeed a nonsmooth convex-nonconcave one. A two stage method is proposed by first smoothing the objective function with a Hermite cubic polynomial approximation, and then obtaining a nearly stationary solution via a recently proposed proximal point gradient descent method. Simulation results show that the proposed framework leads to significant increases in sum secrecy rate and secrecy probability against both the capability-limited adversaries and a hybrid type of adversaries.

TELET: A monotonic algorithm to design large dimensional equiangular tight frames for applications in compressed sensing

An Equiangular tight frame (ETF) - also known as the Welch-bound-equality sequences - consists of a sequence of unit norm vectors whose absolute inner product is identical and minimal. Due to this unique property, these frames are preferred in different applications such as in constructing sensing matrices for compressed sensing systems, robust transmission and quantum computing. Construction of ETFs involves solving a challenging non-convex minimax optimization problem, and only a few methods were successful in constructing them, albeit only for smaller dimensions. In this paper, we propose an iterative algorithm named TEchnique to devise Large dimensional Equiangular Tight-frames (TELET-frames) based on the majorization minimization (MM) procedure - in which we design and minimize a tight upper bound for the ETF cost function at every iteration. Since TELET is designed using the MM approach, it inherits useful properties of MM such as monotonicity and guaranteed convergence to a stationary point. Subsequently, we use the derived frames to construct optimized sensing matrix for compressed sensing systems. In the numerical simulations, we show that the proposed algorithm can generate complex and real frames (in the order of hundreds) with very low mutual coherence value when compared to the state-of-the-art algorithm, with a slight increase in computational cost. Experiments using synthetic data and real images reveal that the optimized sensing matrix obtained through the frames constructed by TELET performs better, in terms of image reconstruction accuracy, than the sensing matrix constructed using state-of-the-art methods.

Global Optimization in Robust Fractional Control of Uncertain Fractional Order Systems: A Thermal Application Using the STM32 Microcontroller

In this paper, we suggest an improvement to our previously undertaken approach. Briefly, this approach consisted of applying the robust fractional predictive control (RFPC) for a class of constrained fractional systems implementing the min–max optimization technique. The RFPC controller requires resolution of a non-convex min–max optimization problem. The resolution of this problem, however, can only conduce to local solutions. The reason is simple: the objective function to be optimized is non-convex due to the presence of uncertainties. In the present work, we propose a global optimization-based RFPC controller for an uncertain fractional order system. A determinist global optimization method, namely, generalized geometric programming (GGP), is proposed to solve this problem for the uncertain fractional order system. The GGP method consists of converting a non-convex problem into a convex one via the application of variable changes. The technique of the convexification of this method is applied in line with the objective function to be optimized. Consequently, we obtained a new convex criterion and a convex problem. From an experimental point of view, we applied the proposed RFPC to a real thermal system using an STM32 microcontroller in order to control our thermal system.

Secure-Reliable Transmission Designs for Full-Duplex Receiver With Finite-Alphabet Inputs

This paper studies a convincingly secure transmission framework under an allowable outage probability for practical finite-alphabet inputs, where a full-duplex receiver (Bob) is taken into account to emit the artificial noise for deteriorating the eavesdropper’s decoding performance. We develop a secure-reliable mechanism to take the place of prior secrecy rate (SR) maximization strategy, where a closed form expression is invoked for substituting the non-closed SR expression upon exploiting the multi-exponential decay fitting approach. Hence, the intractable expectation operation over a large number of noise samples is circumvented. Moreover, a pair of critical probabilities of the reliable transmission and secure outage are first analyzed, and then a low-complexity optimization scheme is formulated. Apart from designing the transmission scheme for classically secure networks consisting of a transmitter, Bob and an eavesdropper, we further carry out an investigation on conceiving a secure-reliable strategy against multiple Eves for improving the system’s extensibility. To this end, analytical expressions of both the reliability outage and secrecy outage probabilities for multiple-Eve scenarios are also derived. Furthermore, a pragmatic iterative solution is conceived for addressing the corresponding max-min optimization problem. Finally, the simulation results validate the significance of our considered secure-reliable transmission in terms of the average SR performance attained.

A Novel Method Designing Optimal Observer Gain for Robust Fault Detection

This paper considers the design of optimal gain of set-value observer for robust fault detection (FD) from a new perspective. Set-based robust FD can be implemented by real-timely check whether the measured output is contained in the corresponding output estimation set or not. Our proposed method focuses on designing observer gains at each step to maximize the exclusion tendency of the measured output from the output estimation zonotope to achieve the goal of robust FD. The key deign logic is formulated as a two-level min-max optimization problem, which is further equivalently transformed into a non-convex single-level programming based on duality theory. Finally, this non-convex single-level programming can be efficiently solved via linear programming and matrix decomposition. At the end, a circuit model is used to illustrate the effectiveness of the proposed method.