Min-max Optimization Problem Research Articles

Learning Universal Adversarial Perturbation by Adversarial Example

Deep learning models have shown to be susceptible to universal adversarial perturbation (UAP), which has aroused wide concerns in the community. Compared with the conventional adversarial attacks that generate adversarial samples at the instance level, UAP can fool the target model for different instances with only a single perturbation, enabling us to evaluate the robustness of the model from a more effective and accurate perspective. The existing universal attack methods fail to exploit the differences and connections between the instance and universal levels to produce dominant perturbations. To address this challenge, we propose a new universal attack method that unifies instance-specific and universal attacks from a feature perspective to generate a more dominant UAP. Specifically, we reformulate the UAP generation task as a minimax optimization problem and then utilize the instance-specific attack method to solve the minimization problem thereby obtaining better training data for generating UAP. At the same time, we also introduce a consistency regularizer to explore the relationship between training data, thus further improving the dominance of the generated UAP. Furthermore, our method is generic with no additional assumptions about the training data and hence can be applied to both data-dependent (supervised) and data-independent (unsupervised) manners. Extensive experiments demonstrate that the proposed method improves the performance by a significant margin over the existing methods in both data-dependent and data-independent settings. Code is available at https://github.com/lisenxd/AT-UAP.

A Generative Model for Texture Synthesis based on Optimal Transport Between Feature Distributions

We propose GOTEX, a general framework for texture synthesis by optimization that constrains the statistical distribution of local features. While our model encompasses several existing texture models, we focus on the case where the comparison between feature distributions relies on optimal transport distances. We show that the semi-dual formulation of optimal transport allows to control the distribution of various possible features, even if these features live in a high-dimensional space. We then study the resulting minimax optimization problem, which corresponds to a Wasserstein generative model, for which the inner concave maximization problem can be solved with standard stochastic gradient methods. The alternate optimization algorithm is shown to be versatile in terms of applications, features and architecture; in particular, it allows to produce high-quality synthesized textures with different sets of features. We analyze the results obtained by constraining the distribution of patches or the distribution of responses to a pre-learned VGG neural network. We show that the patch representation can retrieve the desired textural aspect in a more precise manner. We also provide a detailed comparison with state-of-the-art texture synthesis methods. The GOTEX model based on patch features is also adapted to texture inpainting and texture interpolation. Finally, we show how to use our framework to learn a feed-forward neural network that can synthesize on-the-fly new textures of arbitrary size in a very fast manner. Experimental results and comparisons with the mainstream methods from the literature illustrate the relevance of the generative models learned with GOTEX.

An H∞ approach to data‐driven fault estimation, and isolation for Hammerstein‐Wiener systems

AbstractThis article considers the fault estimation problem in a data‐driven setup for a class of nonlinear systems called Hammerstein‐Wiener. The fault estimation problem is converted to a finite‐horizon minimax optimization problem. To estimate faults, a subspace predictor is utilized to predict the output sequence. Our proposed method is robust against disturbances. The proposed method is briefly expressed as an algorithm that can be implemented by MATLAB software. The suggested algorithm is applied to a numerical MIMO example to verify its validity.

Optimal Design of Tuned Mass-Damper-Inerter for Structure with Uncertain-but-Bounded Parameter

In this study we focus on the H∞ optimization of a tuned mass damper inerter (TMDI), which is implemented on an harmonically forced structure of a single degree of freedom in the presence of stiffness uncertainty. Posed as a min-max optimization problem, its closed-form solutions are analytically derived via an algebraic approach that was newly developed in this work, and ready-to-use formulae of tuning parameters are provided herein for the optimal TMDI (referred to as the TMD). The accuracy of the derived solutions are examined by comparing them with the existing literature and with numerically solved solutions in both deterministic and uncertain scenarios. Our numerical investigation suggested that compared to the classic design, the proposed tuning strategy could effectively reduce the peak vibration amplitude of the host structure in the worst-case scenario. Moreover, its peak vibration amplitude decreases monotonically as the total amount of the tuned mass and inertance increases. Therefore, the incorporation of a grounded inerter in a traditional TMD could render the deteriorated performance of vibration control less important, thereby protecting the primary system against the detuning effect more effectively. Finally, the effectiveness of the proposed design under random excitation is also underlined.

An Optimal Transport Approach to Personalized Federated Learning

Federated learning is a distributed machine learning paradigm, which aims to train a model using the local data of many distributed clients. A key challenge in federated learning is that the data samples across the clients may not be identically distributed. To address this challenge, personalized federated learning with the goal of tailoring the learned model to the data distribution of every individual client has been proposed. In this paper, we focus on this problem and propose a novel personalized Federated Learning scheme based on Optimal Transport (FedOT) as a learning algorithm that learns the optimal transport maps for transferring data points to a common distribution as well as the prediction model under the applied transport map. To formulate the FedOT problem, we extend the standard optimal transport task between two probability distributions to multi-marginal optimal transport problems with the goal of transporting samples from multiple distributions to a common probability domain. We then leverage the results on multi-marginal optimal transport problems to formulate FedOT as a min-max optimization problem and analyze its generalization and optimization properties. We discuss the results of several numerical experiments to evaluate the performance of FedOT under heterogeneous data distributions in federated learning problems.

Unsupervised Learning Discriminative MIG Detectors in Nonhomogeneous Clutter

Principal component analysis (PCA) is a commonly used pattern analysis method that maps high-dimensional data into a lower-dimensional space maximizing the data variance, that results in the promotion of separability of data. Inspired by the principle of PCA, a novel type of learning discriminative matrix information geometry (MIG) detectors in the unsupervised scenario are developed, and applied to signal detection in nonhomogeneous environments. Hermitian positive-definite (HPD) matrices can be used to model the sample data, while the clutter covariance matrix is estimated by the geometric mean of a set of secondary HPD matrices. We define a projection that maps the HPD matrices in a high-dimensional manifold to a low-dimensional and more discriminative one to increase the degree of separation of HPD matrices by maximizing the data variance. Learning a mapping can be formulated as a two-step mini-max optimization problem in Riemannian manifolds, which can be solved by the Riemannian gradient descent algorithm. Three discriminative MIG detectors are illustrated with respect to different geometric measures, i.e., the Log-Euclidean metric, the Jensen–Bregman LogDet divergence and the symmetrized Kullback–Leibler divergence. Simulation results show that performance improvements of the novel MIG detectors can be achieved compared with the conventional detectors and their state-of-the-art counterparts within nonhomogeneous environments.

Stability of bimodal planar switched linear systems with both stable and unstable subsystems

We study dynamics of a bimodal planar linear switched system with a Hurwitz stable and an unstable subsystem. For given flee time from the unstable subsystem, the goal is to find corresponding dwell time in the Hurwitz stable subsystem so that the switched system is asymptotically stable. The dwell-flee relations obtained are in terms of certain smooth functions of the eigenvalues and (generalised) eigenvectors of the subsystem matrices. The results are extended to a special class of symmetric bilinear systems. The results are also extended to a multimodal planar linear switched system in which the switching is governed by an undirected star graph, where the internal node corresponds to a Hurwitz stable (unstable, respectively) subsystem and all the leaves correspond to unstable (Hurwitz stable, respectively) subsystems. For this multimodal system, dwell-flee relations are obtained as solutions of a minimax optimisation problem.

Design of maximum-stability PID controllers for LTI systems based on a stabilizing-set construction method

BackgroundThe PID algorithm has been widely used in control of chemical processes. A maximum-stability PID controller can provide superior stability robustness towards plant's variations and maximum exponential decay rate for disturbance rejection. However, design of maximum-stability PID controller is a min-max optimization problem and an effective method is still lack in the literature. MethodsBased on the characterization of stabilizing PID controller set, an efficient algorithm is developed to test if a plant is σ-stabilizable, where σ is abscissa or stability degree of the Hurwitz stable closed-loop characteristic polynomial. This algorithm is then used along with a bisection strategy to find a σ-interval [σε*,σε*+ε] which contains the maximum stability degree σ* for a specified ε, and the PID controller parameter set for achieving the stability degree σε*. Significant findingsThis paper has presented a systematic and efficient approach to design PID controllers with maximum degree of stability. The principal results include: (i) an improved theorem is presented for identifying stabilizing kp-intervals such that unnecessary computations are avoided; (ii) a simple yet effective method has been adopted to provide a non-conservative interval of σ which facilitates the bisectional branch-and-bound operation; (iii) the design procedure does not involve the actual construction of stabilizing PID controller sets thus renders its efficiency.

Topological synthesis of fluidic pressure-actuated robust compliant mechanisms

This paper presents a robust density-based topology optimization approach for synthesizing pressure-actuated compliant mechanisms. To ensure functionality under manufacturing inaccuracies, the robust or three-field formulation is employed, involving dilated, intermediate and eroded realizations of the design. Darcy’s law in conjunction with a conceptualized drainage term is used to model the pressure load as a function of the design vector. The consistent nodal loads are evaluated from the obtained pressure field using the standard finite element method. The objective and load sensitivities are obtained using the adjoint-variable approach. A multi-criteria objective involving both the stiffness and flexibility of the mechanism is employed in the robust formulation, and min–max optimization problems are solved to obtain pressure-actuated inverter, gripper, and contractor compliant mechanisms with different minimum feature sizes. Limitations of the linear elasticity assumptions while designing mechanisms are identified with high pressure loads. Challenges involved in designing finite deformable pressure-actuated compliant mechanisms are presented.

Wide-area damping power system control based on robust self-triggered model predictive control

Unavoidable random disturbances during the operation of large-scale interconnected power systems may undermine their stability. To prevent this problem, we propose a damping control method based on robust self-triggered model predictive control. First, we linearize the power system model and determine the controller installation location and feedback signal through geometric measurements. Subsequently, the model order is reduced by applying Schur method for balanced truncation. According to the assumed distribution and boundary range of random disturbance, the multi-cell robust predictive control minimax optimization problem described by the vertex is established. By deriving the upper bound of the maximum problem, the minimax problem is transformed into minimizing the upper bound problem to deal with the uncertain problem, which is transformed into linear matrix inequality for controller design. Finally, we introduce a self-triggered control strategy that avoids solving a constrained optimization problem at every time step. This strategy reduces the number of controller responses and resource requirements for sampling and communication, thereby improving computational efficiency. Selecting the two zone four-machine system as the test system, the simulation results demonstrate that the proposed control strategy can effectively eliminate the effect of random disturbances on the stable operation of power systems, improve the damping characteristics of power systems, and suppress low-frequency oscillations.

Generative Adversarial Training for Supervised and Semi-supervised Learning.

Neural networks have played critical roles in many research fields. The recently proposed adversarial training (AT) can improve the generalization ability of neural networks by adding intentional perturbations in the training process, but sometimes still fail to generate worst-case perturbations, thus resulting in limited improvement. Instead of designing a specific smoothness function and seeking an approximate solution used in existing AT methods, we propose a new training methodology, named Generative AT (GAT) in this article, for supervised and semi-supervised learning. The key idea of GAT is to formulate the learning task as a minimax game, in which the perturbation generator aims to yield the worst-case perturbations that maximize the deviation of output distribution, while the target classifier is to minimize the impact of this perturbation and prediction error. To solve this minimax optimization problem, a new adversarial loss function is constructed based on the cross-entropy measure. As a result, the smoothness and confidence of the model are both greatly improved. Moreover, we develop a trajectory-preserving-based alternating update strategy to enable the stable training of GAT. Numerous experiments conducted on benchmark datasets clearly demonstrate that the proposed GAT significantly outperforms the state-of-the-art AT methods in terms of supervised and semi-supervised learning tasks, especially when the number of labeled examples is rather small in semi-supervised learning.

On approximate optimality conditions for robust multi-objective convex optimization problems

In this paper, we are interested in the study of approximate optimality conditions for weakly ϵ-efficient solutions to robust multi-objective optimization problems ((RMOP) for short) in view of its associated minimax optimization problem (MMOP). To this end, we first establish the relationship between a weakly ϵ-efficient solution to the problem (RMOP) and an α-solution to the problem (MMOP), where and . Then, we explore the representation of the so-called β-normal set (where is a given parameter) to a closed convex set at some reference point by two methods. At last, by employing the α-subdifferential of the max-function and the obtained representation of the β-normal set, we establish an approximate necessary optimality condition for the problem (RMOP). Moreover, we also give an example to illustrate our results.

A novel privacy-aware model for nonparametric decentralized detection

In recent years, the increasing development of the Internet of Things (IoT) demands the enhancement of both network security and user privacy protection. In the decentralized IoT network, multiple sensors send local observations to a fusion center for data aggregation and authorized hypothesis detection. But at the same time, private information might be inferred illegally, which would cause privacy leakage. In this paper, a novel privacy-aware model named AL-UP is proposed for the nonparametric decentralized detection in the IoT network. It aims to design a local differential privacy and data projection based sanitization mechanism for sensors, to hide the sensitive information in raw observations and protect the data and inference privacy. Based on the adversarial learning framework and linear discriminant analysis, we propose a max-min optimization problem to design parameters of the sanitization mechanism and hypothesis detection rules. The problem is solved via the block coordinate descend method. Numerical results on various public datasets indicate that the proposed model achieves better utility-privacy trade-off than the state of the arts.

Secrecy‐oriented user association in ultra dense heterogeneous networks against strategically colluding adversaries

Network densification is recognized as the key technology to meet the ever growing demand of data traffic in next generation wireless networks. However the proliferation of small cell base stations (SCBSs) introduces vulnerabilities to information security, as they are prone to eavesdropping attacks. In this paper, it is studied how user association strategies can be specifically tailored to meet this security challenge in a ultra dense heterogeneous cellular network(UDHCN) with a group of colluding eavesdroppers. In particular, the situation is considered where eavesdroppers may have limited capabilities in intercepting and decoding data traffic. In this setting each eavesdropper has to make its own decision on eavesdropping targets, and they are able to take concerted actions to sabotage information security. To address this challenge, a zero sum game framework to reflect the conflicting interests of users and eavesdroppers is proposed and a user association scheme is devised that aims to maximize system sum secrecy rate against these adversaries whose actions intend to minimize sum secrecy rate. The case that all adversaries have limited eavesdropping capabilities is first considered, and it is shown that the corresponding zero sum game is indeed a bilinear game, and its Nash equilibrium solution can be readily approximated using a saddle point Frank Wolfe(SP-FW)algorithm. Then the framework is extended to the case where adversaries with a diverse configuration of eavesdropping capabilities exist. In this case, it is shown that the underlying minimax optimization problem is indeed a nonsmooth convex-nonconcave one. A two stage method is proposed by first smoothing the objective function with a Hermite cubic polynomial approximation, and then obtaining a nearly stationary solution via a recently proposed proximal point gradient descent method. Simulation results show that the proposed framework leads to significant increases in sum secrecy rate and secrecy probability against both the capability-limited adversaries and a hybrid type of adversaries.

TELET: A monotonic algorithm to design large dimensional equiangular tight frames for applications in compressed sensing

An Equiangular tight frame (ETF) - also known as the Welch-bound-equality sequences - consists of a sequence of unit norm vectors whose absolute inner product is identical and minimal. Due to this unique property, these frames are preferred in different applications such as in constructing sensing matrices for compressed sensing systems, robust transmission and quantum computing. Construction of ETFs involves solving a challenging non-convex minimax optimization problem, and only a few methods were successful in constructing them, albeit only for smaller dimensions. In this paper, we propose an iterative algorithm named TEchnique to devise Large dimensional Equiangular Tight-frames (TELET-frames) based on the majorization minimization (MM) procedure - in which we design and minimize a tight upper bound for the ETF cost function at every iteration. Since TELET is designed using the MM approach, it inherits useful properties of MM such as monotonicity and guaranteed convergence to a stationary point. Subsequently, we use the derived frames to construct optimized sensing matrix for compressed sensing systems. In the numerical simulations, we show that the proposed algorithm can generate complex and real frames (in the order of hundreds) with very low mutual coherence value when compared to the state-of-the-art algorithm, with a slight increase in computational cost. Experiments using synthetic data and real images reveal that the optimized sensing matrix obtained through the frames constructed by TELET performs better, in terms of image reconstruction accuracy, than the sensing matrix constructed using state-of-the-art methods.

Global Optimization in Robust Fractional Control of Uncertain Fractional Order Systems: A Thermal Application Using the STM32 Microcontroller

In this paper, we suggest an improvement to our previously undertaken approach. Briefly, this approach consisted of applying the robust fractional predictive control (RFPC) for a class of constrained fractional systems implementing the min–max optimization technique. The RFPC controller requires resolution of a non-convex min–max optimization problem. The resolution of this problem, however, can only conduce to local solutions. The reason is simple: the objective function to be optimized is non-convex due to the presence of uncertainties. In the present work, we propose a global optimization-based RFPC controller for an uncertain fractional order system. A determinist global optimization method, namely, generalized geometric programming (GGP), is proposed to solve this problem for the uncertain fractional order system. The GGP method consists of converting a non-convex problem into a convex one via the application of variable changes. The technique of the convexification of this method is applied in line with the objective function to be optimized. Consequently, we obtained a new convex criterion and a convex problem. From an experimental point of view, we applied the proposed RFPC to a real thermal system using an STM32 microcontroller in order to control our thermal system.

Secure-Reliable Transmission Designs for Full-Duplex Receiver With Finite-Alphabet Inputs

This paper studies a convincingly secure transmission framework under an allowable outage probability for practical finite-alphabet inputs, where a full-duplex receiver (Bob) is taken into account to emit the artificial noise for deteriorating the eavesdropper’s decoding performance. We develop a secure-reliable mechanism to take the place of prior secrecy rate (SR) maximization strategy, where a closed form expression is invoked for substituting the non-closed SR expression upon exploiting the multi-exponential decay fitting approach. Hence, the intractable expectation operation over a large number of noise samples is circumvented. Moreover, a pair of critical probabilities of the reliable transmission and secure outage are first analyzed, and then a low-complexity optimization scheme is formulated. Apart from designing the transmission scheme for classically secure networks consisting of a transmitter, Bob and an eavesdropper, we further carry out an investigation on conceiving a secure-reliable strategy against multiple Eves for improving the system’s extensibility. To this end, analytical expressions of both the reliability outage and secrecy outage probabilities for multiple-Eve scenarios are also derived. Furthermore, a pragmatic iterative solution is conceived for addressing the corresponding max-min optimization problem. Finally, the simulation results validate the significance of our considered secure-reliable transmission in terms of the average SR performance attained.

A Novel Method Designing Optimal Observer Gain for Robust Fault Detection

This paper considers the design of optimal gain of set-value observer for robust fault detection (FD) from a new perspective. Set-based robust FD can be implemented by real-timely check whether the measured output is contained in the corresponding output estimation set or not. Our proposed method focuses on designing observer gains at each step to maximize the exclusion tendency of the measured output from the output estimation zonotope to achieve the goal of robust FD. The key deign logic is formulated as a two-level min-max optimization problem, which is further equivalently transformed into a non-convex single-level programming based on duality theory. Finally, this non-convex single-level programming can be efficiently solved via linear programming and matrix decomposition. At the end, a circuit model is used to illustrate the effectiveness of the proposed method.

Polytopic Robust Distributed Kalman Consensus Filter for Sensor Networks

Distributed estimation is a fundamental task performed by sensor networks, often achieved through the Kalman consensus filter. However, it requires exact models, which is seldom possible in practice since parametric uncertainties are usually unavoidable. We address in this paper the robust distributed filtering problem regarding linear discrete-time systems subject to polytopic uncertainties. We consider uncertainties in all parameter matrices of the target system and sensing models. First, we introduce a centralized filter, obtained as the solution to a min-max optimization problem whose cost function collectively weights all the underlying polytope vertices. Then, we derive a fully distributed version of this filter by applying the hybrid consensus on measurements and information approach. The estimators are recursive and do not rely on numerical solvers, appealing features for real-time applications. We validate our approach and assess its performance with a numerical example, comparing it with other robust distributed strategies.

T-S fuzzy-model-based adaptive cruise control for longitudinal car-following considering vehicle lateral stability

Adaptive cruise control is one of the essential technologies of advanced driver assistance systems, which is used to maintain a safe distance between an ego vehicle and a preceding vehicle and has been extensively applied in the automotive industry and control community. Note that some vehicle manoeuvres may approach handling limits to prevent collisions under complex road conditions, which often leads to vehicle lateral instability while cruising. In this study, a T-S fuzzy model predictive control framework is applied to the problem of adaptive cruise control. Variations in the preceding vehicle velocity and road surface conditions are considered to formulate adaptive cruise control as a tracking control problem of a T-S fuzzy system subject to parameter uncertainties and external persistent perturbations. Then, a robust positively invariant set is introduced to derive an admissible T-S fuzzy controller by solving a min-max optimization problem under a series of linear matrix inequality constraints. Finally, a CarSim/MATLAB joint simulation is conducted to illustrate the effectiveness of the proposed method, which ensures longitudinal adaptive cruise control for a car-following scenario with lateral vehicle stability.