Minimax Optimization Research Articles

Decentralized Riemannian Algorithm for Nonconvex Minimax Problems

The minimax optimization over Riemannian manifolds (possibly nonconvex constraints) has been actively applied to solve many problems, such as robust dimensionality reduction and deep neural networks with orthogonal weights (Stiefel manifold). Although many optimization algorithms for minimax problems have been developed in the Euclidean setting, it is difficult to convert them into Riemannian cases, and algorithms for nonconvex minimax problems with nonconvex constraints are even rare. On the other hand, to address the big data challenges, decentralized (serverless) training techniques have recently been emerging since they can reduce communications overhead and avoid the bottleneck problem on the server node. Nonetheless, the algorithm for decentralized Riemannian minimax problems has not been studied. In this paper, we study the distributed nonconvex-strongly-concave minimax optimization problem over the Stiefel manifold and propose both deterministic and stochastic minimax methods. The Steifel manifold is a non-convex set. The global function is represented as the finite sum of local functions. For the deterministic setting, we propose DRGDA and prove that our deterministic method achieves a gradient complexity of O( epsilon(-2)) under mild conditions. For the stochastic setting, we propose DRSGDA and prove that our stochastic method achieves a gradient complexity of O( epsilon(-4)). The DRGDA and DRSGDA are the first algorithms for distributed minimax optimization with nonconvex constraints with exact convergence. Extensive experimental results on the Deep Neural Networks (DNNs) training over the Stiefel manifold demonstrate the efficiency of our algorithms.

Offline Imitation Learning with Suboptimal Demonstrations via Relaxed Distribution Matching

Offline imitation learning (IL) promises the ability to learn performant policies from pre-collected demonstrations without interactions with the environment. However, imitating behaviors fully offline typically requires numerous expert data. To tackle this issue, we study the setting where we have limited expert data and supplementary suboptimal data. In this case, a well-known issue is the distribution shift between the learned policy and the behavior policy that collects the offline data. Prior works mitigate this issue by regularizing the KL divergence between the stationary state-action distributions of the learned policy and the behavior policy. We argue that such constraints based on exact distribution matching can be overly conservative and hamper policy learning, especially when the imperfect offline data is highly suboptimal. To resolve this issue, we present RelaxDICE, which employs an asymmetrically-relaxed f-divergence for explicit support regularization. Specifically, instead of driving the learned policy to exactly match the behavior policy, we impose little penalty whenever the density ratio between their stationary state-action distributions is upper bounded by a constant. Note that such formulation leads to a nested min-max optimization problem, which causes instability in practice. RelaxDICE addresses this challenge by supporting a closed-form solution for the inner maximization problem. Extensive empirical study shows that our method significantly outperforms the best prior offline IL method in six standard continuous control environments with over 30% performance gain on average, across 22 settings where the imperfect dataset is highly suboptimal.

Adversarial Word Dilution as Text Data Augmentation in Low-Resource Regime

Data augmentation is widely used in text classification, especially in the low-resource regime where a few examples for each class are available during training. Despite the success, generating data augmentations as hard positive examples that may increase their effectiveness is under-explored. This paper proposes an Adversarial Word Dilution (AWD) method that can generate hard positive examples as text data augmentations to train the low-resource text classification model efficiently. Our idea of augmenting the text data is to dilute the embedding of strong positive words by weighted mixing with unknown-word embedding, making the augmented inputs hard to be recognized as positive by the classification model. We adversarially learn the dilution weights through a constrained min-max optimization process with the guidance of the labels. Empirical studies on three benchmark datasets show that AWD can generate more effective data augmentations and outperform the state-of-the-art text data augmentation methods. The additional analysis demonstrates that the data augmentations generated by AWD are interpretable and can flexibly extend to new examples without further training.

Scaling Marginalized Importance Sampling to High-Dimensional State-Spaces via State Abstraction

We consider the problem of off-policy evaluation (OPE) in reinforcement learning (RL), where the goal is to estimate the performance of an evaluation policy, pie, using a fixed dataset, D, collected by one or more policies that may be different from pie. Current OPE algorithms may produce poor OPE estimates under policy distribution shift i.e., when the probability of a particular state-action pair occurring under pie is very different from the probability of that same pair occurring in D. In this work, we propose to improve the accuracy of OPE estimators by projecting the high-dimensional state-space into a low-dimensional state-space using concepts from the state abstraction literature. Specifically, we consider marginalized importance sampling (MIS) OPE algorithms which compute state-action distribution correction ratios to produce their OPE estimate. In the original ground state-space, these ratios may have high variance which may lead to high variance OPE. However, we prove that in the lower-dimensional abstract state-space the ratios can have lower variance resulting in lower variance OPE. We then highlight the challenges that arise when estimating the abstract ratios from data, identify sufficient conditions to overcome these issues, and present a minimax optimization problem whose solution yields these abstract ratios. Finally, our empirical evaluation on difficult, high-dimensional state-space OPE tasks shows that the abstract ratios can make MIS OPE estimators achieve lower mean-squared error and more robust to hyperparameter tuning than the ground ratios.

Nodal integration-based particle finite element method (N-PFEM) for poro-elastoplastic modelling of saturated soils under large deformation

This paper presents the nodal integration-based particle finite element method (N-PFEM) for poro-elastoplastic analysis of saturated soils subject to large deformation, utilising the generalised Hellinger-Reissner variational principle to reformulate the governing equations for saturated soil dynamics into a min–max optimisation problem. With finite element discretisation and nodal integration over cells, the problem is transformed into a standard second-order cone programming problem, efficiently resolved using the advanced primal–dual interior point method. The N-PFEM method has several advantages, including the use of linear triangular elements without volumetric locking issues, the avoidance of regularisation techniques, and the elimination of tedious variable mapping after remeshing. The numerical model is validated for large deformation analysis of saturated soils with a series of benchmarks against available analytical and numerical solutions, with a case study of the deformation of an embankment considering stone column reinforcement also carried out. This N-PFEM framework offers an effective and efficient simulation approach for the evolutionary behaviour of saturated soils with large deformation in complex geotechnical configurations of practical relevance.

Friedrichs Learning: Weak Solutions of Partial Differential Equations via Deep Learning

This paper proposes Friedrichs learning as a novel deep learning methodology that can learn the weak solutions of PDEs via a minimax formulation, which transforms the PDE problem into a minimax optimization problem to identify weak solutions. The name “Friedrichs learning” is to highlight the close relation between our learning strategy and Friedrichs theory on symmetric systems of PDEs. The weak solution and the test function in the weak formulation are parameterized as deep neural networks in a mesh-free manner, which are alternately updated to approach the optimal solution networks approximating the weak solution and the optimal test function, respectively. Extensive numerical results indicate that our mesh-free Friedrichs learning method can provide reasonably good solutions for a wide range of PDEs defined on regular and irregular domains, where conventional numerical methods such as finite difference methods and finite element methods may be tedious or difficult to be applied, especially for those with discontinuous solutions in high-dimensional problems.

Two-timescale recurrent neural networks for distributed minimax optimization

In this paper, we present two-timescale neurodynamic optimization approaches to distributed minimax optimization. We propose four multilayer recurrent neural networks for solving four different types of generally nonlinear convex–concave minimax problems subject to linear equality and nonlinear inequality constraints. We derive sufficient conditions to guarantee the stability and optimality of the neural networks. We demonstrate the viability and efficiency of the proposed neural networks in two specific paradigms for Nash-equilibrium seeking in a zero-sum game and distributed constrained nonlinear optimization.

Distributionally robust unsupervised domain adaptation

Obtaining ground-truth label information from real-world data along with uncertainty quantification can be difficult or even infeasible. In the absence of labeled data for a certain task, unsupervised domain adaptation (UDA) techniques have shown great accomplishment by learning transferable knowledge from labeled source domain data and applying it to unlabeled target domain data, yet scarce studies consider addressing uncertainties under domain shifts to improve the model robustness. Distributionally robust learning (DRL) is emerging as a high-potential technique for building reliable learning systems that are robust to distribution shifts. In this study, we propose a distributionally robust unsupervised domain adaptation (DRUDA) method to enhance the machine learning model generalization ability under input space perturbations. The DRL-based UDA learning scheme is formulated as a min–max optimization problem by optimizing worst-case perturbations of the training source data. Our Wasserstein distributionally robust framework can reduce the shifts in the joint distributions across domains. The proposed DRUDA has been tested on digit datasets and the Office-31 dataset, compared with other state-of-the-art domain adaptation techniques. Our experimental results show that the proposed DRUDA leads to improvements in domain adaptation accuracy performance on target domains.

Latency minimization for multiuser computation offloading in fog-radio access networks

Recently, the Fog-Radio Access Network (F-RAN) has gained considerable attention, because of its flexible architecture that allows rapid response to user requirements. In this paper, computational offloading in F-RAN is considered, where multiple User Equipments (UEs) offload their computational tasks to the F-RAN through fog nodes. Each UE can select one of the fog nodes to offload its task, and each fog node may serve multiple UEs. The tasks is computed by the fog nodes or further offloaded to the cloud via a capacity-limited fronhaul link. In order to compute all UEs' tasks quickly, joint optimization of UE-Fog association, radio and computation resources of F-RAN is proposed to minimize the maximum latency of all UEs. This min-max problem is formulated as a Mixed Integer Nonlinear Program (MINP). To tackle it, first, MINP is reformulated as a continuous optimization problem, and then the Majorization Minimization (MM) method is used to find a solution. The MM approach that we develop is unconventional in that each MM subproblem is solved inexactly with the same provable convergence guarantee as the exact MM, thereby reducing the complexity of MM iteration. In addition, a cooperative offloading model is considered, where the fog nodes compress-and-forward their received signals to the cloud. Under this model, a similar min-max latency optimization problem is formulated and tackled by the inexact MM. Simulation results show that the proposed algorithms outperform some offloading strategies, and that the cooperative offloading can exploit transmission diversity better than noncooperative offloading to achieve better latency performance.

Stability for Constrained Minimax Optimization

Stability for Constrained Minimax Optimization

Robust Beamforming Design for Dual-Function Radar-Communication System

In this paper, we investigate the problem of robust beamforming for multiple-input multiple-output (MIMO) dual-function radar-communication (DFRC) system, when there exist radar look direction mismatch and communication channel state information (CSI) error. To address these system imperfections, a region-of-interest (ROI) rather than a specific angle is considered for radar sensing, and the communication CSI is assumed to be available with a bounded unknown error. With these considerations, we design a robust dual-function beamformer through optimizing the radar sensing performance in terms of the energy radiated towards the ROI, while guaranteeing the communication quality-of-service (QoS) measured by the per-user signal-to-interference-plus-noise ratio (SINR) as required. For this robust design, a max-min optimization problem is formulated and a semidefinite programming (SDP) reformulation based on the S-procedure is presented. The popular semidefinite relaxation (SDR) technique is employed to solve the resulting non-convex problem, and the tightness of SDR is discussed. Further, we propose to incorporate an extra constraint of main beam control into the problem to achieve a flat magnitude response of the beampattern over the ROI. In addition, it is shown that the proposed approach is straightforwardly applicable to the scenario of multiple targets. Numerical examples are carried out to show the tightness, and demonstrate the effectiveness and robustness of the proposed MIMO-DFRC beamforming approach.

Robust energy management in isolated microgrids with hydrogen storage and demand response

Hydrogen is called to be one of the main energy vectors in future energy systems, especially for energy storage, where this carrier presents some interesting features. Its application in microgrids might help to improve economic, environmental and reliability indicators, providing greater storage capacity than other technologies, like batteries. Following this idea, this paper develops a robust energy management methodology for isolated microgrids considering hydrogen storage and demand response initiatives. The new proposal is raised as a nested max-min optimization framework. To reduce the original problem to a tractable single-level one, a master-slave scheme is developed by which binary variables can be fixed and thus the inner problem can be reduced to its equivalent Karush-Kuhn-Tucker conditions. Then, the overall optimization paradigm is solved in an iterative fashion used the Constraint-and-Column Generation Algorithm. The resulting tool is applied to a benchmark isolated microgrid, thus validating it for industry applications. Moreover, different relevant results are analysed in-depth. Thus, the role of hydrogen storage and demand response initiatives is discussed, concluding that flexible demand has a more notable impact on monetary savings than hydrogen storage, reducing the total cost by 6 % with respect to the base case. Some intrinsic issues are also identified. For example, it is observed that flexible consumers are more frequently called when the hydrogen chain is enabled, which may provoke undesirable effects like response fatigue.

Deep neural network classifier for multidimensional functional data

AbstractWe propose a new approach, called as functional deep neural network (FDNN), for classifying multidimensional functional data. Specifically, a deep neural network is trained based on the principal components of the training data which shall be used to predict the class label of a future data function. Unlike the popular functional discriminant analysis approaches which only work for one‐dimensional functional data, the proposed FDNN approach applies to general non‐Gaussian multidimensional functional data. Moreover, when the log density ratio possesses a locally connected functional modular structure, we show that FDNN achieves minimax optimality. The superiority of our approach is demonstrated through both simulated and real‐world datasets.

Feature optimization method for white feather broiler health monitoring technology

The existing research on white feather broiler health monitoring technology concentrate on the selection of classification algorithms and optimization of internal parameters, ignoring the white feather broiler sound signal data set (abbr. signal data set) and internal features for training classifiers. In this paper, the authors shifted the research focus to the signal data set, and proposed an applicable feature optimization method in order to improve the practicality and stability of broiler health monitoring technology. First, for outliers in the signal data set, the boxplot was used to detect them, and they were transformed into missing values. For all missing values in the signal data set, six missing value processing methods were used to process them, respectively. The optimal decision tree classifier (abbr. DT classifier) filling was determined. Second, three normalization methods were used to process the signal data set, respectively, and the optimal min–max normalization method was determined. Then, the Pearson correlation coefficient and the p-value were used to perform correlation tests on features and labels in the signal data set, then the linear–nonlinear-fusion-based (abbr. LNLF-based) feature selection method was newly proposed to select those features that have a great influence on the classification effect of the random forest classifier (abbr. RF classifier). Multiple tests had verified its superiority. Finally, ten white feather broiler sound signal validation data sets (abbr. validation data sets) were used to verify the proposed feature optimization method. It shows that the average classification accuracy achieved by the RF classifier on the signal data set before and after feature optimization improved from 72.85% to 85.28%, with an improvement of 12.43%. The average G-mean achieved by the RF classifier improved from 0.828 to 0.939, with an improvement of 0.111. The average classification accuracy and average G-mean achieved by the RF classifier on the validation data set before and after feature optimization improved significantly, with an average improvement of 13.26% and 0.093, respectively. This method improves the reliability of the existing classifiers in another way, which is an important supplement to the feature engineering study, and promotes the research progress of the existing white feather broiler health monitoring technology.

Resource allocation and BER performance analysis of NOMA based cooperative networks

Resource sharing and management can significantly improve the performance and spectral efficiency of wireless systems. In power domain non-orthogonal multiple access (NOMA) systems, users share a common bandwidth for simultaneous data transmission and therfore, the spectral efficiency of the system will be improved at the expense of increased complexity. Provided proper power allocation, it is possible to detect symbols at receivers. In this paper, the downlink of a cellular system is considered where a base station (BS) communicates with two users using NOMA with the assistance of a decode-and-forward (DF) relay. In the proposed scheme, receivers combine received signals from direct and cooperative links, and decode symbols by employing successive interference cancellation (SIC). The bit error rate (BER) performance of the proposed scheme is analyzed and the optimal power allocation is also derived through a Min–Max optimization, and a closed form approximate solution is proposed for binary phase shift keying (BPSK) modulation. Furthermore, it is proved that the proposed receiver achieves full diversity order for all users for proper choice of power allocation. Simulation results also corroborate BER and diversity analysis.

Robust Data-Enabled Predictive Control: Tractable Formulations and Performance Guarantees

We introduce a general framework for robust data-enabled predictive control (DeePC) for linear time-invariant (LTI) systems, which enables us to obtain robust and optimal control in a receding-horizon fashion based on inexact input and output data. Robust DeePC solves a min-max optimization problem to compute the optimal control sequence that is resilient to all possible realizations of the uncertainties in data within a prescribed uncertainty set. We present computationally tractable reformulations of the min-max problem with various uncertainty sets. Moreover, we show that even though an accurate prediction of the future behavior is unattainable due to inaccessibility of exact data, the obtained control sequence provides performance guarantees for the actually <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">realized</i> input and output cost in open loop. Finally, we demonstrate the performance of robust DeePC using high-fidelity simulations of a power converter system.

Learning-Based Event-Triggered Tracking Control for Nonlinear Networked Control Systems With Unmatched Disturbance

This article concentrates on optimal tracking control for a class of nonlinear networked systems subjecting to limited network bandwidth and unmatched disturbance. Given the models of the control and reference systems, the considered optimal tracking control issue is initially formulated as a minimax optimization problem. Then, with the introduction of an event-triggered mechanism used for saving bandwidth, the formulated problem is transformed into solving an event-based Hamilton-Jacobi-Isaacs (HJI) equation by recurring to the Bellman optimality theory. Based on the HJI equation, we demonstrate that the stability of the concerned system in the sense of uniformly ultimately bounded (UUB) can be guaranteed with the envisioned optimal control and worst disturbance policies. Here, the disturbance policy can be varied periodically while the control policy can only be updated at event-triggering instants, which differs from the existed researches. Furthermore, we propose a reinforcement learning (RL)-based algorithm to handle the constructed HJI equation and thus settle the studied tracking control problem. The effectiveness of the algorithm is finally validated by both theoretical analysis and simulations.

Distributed robust moving horizon estimation for multisensor systems with stochastic and norm‐bounded uncertainties

SummaryIn this study, a distributed robust moving horizon estimation (DRMHE) approach is proposed for time‐varying multisensor systems affected by stochastic and norm‐bounded uncertainties. First, by considering the uncertainties in all model matrices, stochastic min–max optimization problems in local and collective forms are presented to achieve the proposed estimator. Then, by converting the problems to robust regularized least‐squares problems with uncertain parameters, closed‐form solutions are obtained for estimations in both local and collective forms. Furthermore, the stability of the proposed estimator is investigated under some appropriate conditions. At last, two different examples are employed to show the robust performance and superiority of the proposed DRMHE approach compared to the existing methods.

Breaking the Sample Size Barrier in Model-Based Reinforcement Learning with a Generative Model

This paper studies a central issue in modern reinforcement learning, the sample efficiency, and makes progress toward solving an idealistic scenario that assumes access to a generative model or a simulator. Despite a large number of prior works tackling this problem, a complete picture of the trade-offs between sample complexity and statistical accuracy has yet to be determined. In particular, all prior results suffer from a severe sample size barrier in the sense that their claimed statistical guarantees hold only when the sample size exceeds some enormous threshold. The current paper overcomes this barrier and fully settles this problem; more specifically, we establish the minimax optimality of the model-based approach for any given target accuracy level. To the best of our knowledge, this work delivers the first minimax-optimal guarantees that accommodate the entire range of sample sizes (beyond which finding a meaningful policy is information theoretically infeasible).

Robust expansion planning and hardening strategy of meshed multi-energy distribution networks for resilience enhancement

As an integrated energy system (IES) incorporated with electricity, gas, and heating networks, the meshed multi-energy distribution network (MMDN) can operate more economically with higher energy utilization efficiency. To achieve high reliability, MMDN is mesh-constructed and radial-operated, complicating the operation scenarios involved in planning. Existing coordinated planning models seldom consider network reconfiguration and hardening strategies collaboratively for extreme contingencies, which is not consistent with actual conditions. In addition, considering the multi-stage discreteness of the expansion planning problem, the existing robust algorithms may feedback invalid cuts. The neglect of such issues could introduce a momentous impact on the optimality of solutions. This paper presents a resiliency-oriented expansion planning and hardening model of MMDN, and the hierarchical algorithm is developed. Specifically, the model is formulated as a min–max-min optimization problem including investment and operation levels. The investment level problem optimizes planning schemes constrained with the robustness verification of the operation level problem. The operation level problem examines the operational cost and feeds back iterative information to the investment level. An additional cuts generation method is developed in the operation level, which effectively deals with invalid feedback information caused by the existing binary variable parameterization. Furthermore, accelerating strategies including infeasible region reduction, multi-cut feedback, and contingency set reduction are developed. To be solved by off-the-shelf solvers, the model is cast as a mixed-integer linear programming (MILP) problem by adopting the convex hull relaxation and linearization techniques for operational constraints with high accuracy. Numerical test results justify the effectiveness of the model.