Minimax Optimization Research Articles

Scaling Marginalized Importance Sampling to High-Dimensional State-Spaces via State Abstraction

We consider the problem of off-policy evaluation (OPE) in reinforcement learning (RL), where the goal is to estimate the performance of an evaluation policy, pie, using a fixed dataset, D, collected by one or more policies that may be different from pie. Current OPE algorithms may produce poor OPE estimates under policy distribution shift i.e., when the probability of a particular state-action pair occurring under pie is very different from the probability of that same pair occurring in D. In this work, we propose to improve the accuracy of OPE estimators by projecting the high-dimensional state-space into a low-dimensional state-space using concepts from the state abstraction literature. Specifically, we consider marginalized importance sampling (MIS) OPE algorithms which compute state-action distribution correction ratios to produce their OPE estimate. In the original ground state-space, these ratios may have high variance which may lead to high variance OPE. However, we prove that in the lower-dimensional abstract state-space the ratios can have lower variance resulting in lower variance OPE. We then highlight the challenges that arise when estimating the abstract ratios from data, identify sufficient conditions to overcome these issues, and present a minimax optimization problem whose solution yields these abstract ratios. Finally, our empirical evaluation on difficult, high-dimensional state-space OPE tasks shows that the abstract ratios can make MIS OPE estimators achieve lower mean-squared error and more robust to hyperparameter tuning than the ground ratios.

Nodal integration-based particle finite element method (N-PFEM) for poro-elastoplastic modelling of saturated soils under large deformation

This paper presents the nodal integration-based particle finite element method (N-PFEM) for poro-elastoplastic analysis of saturated soils subject to large deformation, utilising the generalised Hellinger-Reissner variational principle to reformulate the governing equations for saturated soil dynamics into a min–max optimisation problem. With finite element discretisation and nodal integration over cells, the problem is transformed into a standard second-order cone programming problem, efficiently resolved using the advanced primal–dual interior point method. The N-PFEM method has several advantages, including the use of linear triangular elements without volumetric locking issues, the avoidance of regularisation techniques, and the elimination of tedious variable mapping after remeshing. The numerical model is validated for large deformation analysis of saturated soils with a series of benchmarks against available analytical and numerical solutions, with a case study of the deformation of an embankment considering stone column reinforcement also carried out. This N-PFEM framework offers an effective and efficient simulation approach for the evolutionary behaviour of saturated soils with large deformation in complex geotechnical configurations of practical relevance.

Friedrichs Learning: Weak Solutions of Partial Differential Equations via Deep Learning

This paper proposes Friedrichs learning as a novel deep learning methodology that can learn the weak solutions of PDEs via a minimax formulation, which transforms the PDE problem into a minimax optimization problem to identify weak solutions. The name “Friedrichs learning” is to highlight the close relation between our learning strategy and Friedrichs theory on symmetric systems of PDEs. The weak solution and the test function in the weak formulation are parameterized as deep neural networks in a mesh-free manner, which are alternately updated to approach the optimal solution networks approximating the weak solution and the optimal test function, respectively. Extensive numerical results indicate that our mesh-free Friedrichs learning method can provide reasonably good solutions for a wide range of PDEs defined on regular and irregular domains, where conventional numerical methods such as finite difference methods and finite element methods may be tedious or difficult to be applied, especially for those with discontinuous solutions in high-dimensional problems.

Two-timescale recurrent neural networks for distributed minimax optimization

In this paper, we present two-timescale neurodynamic optimization approaches to distributed minimax optimization. We propose four multilayer recurrent neural networks for solving four different types of generally nonlinear convex–concave minimax problems subject to linear equality and nonlinear inequality constraints. We derive sufficient conditions to guarantee the stability and optimality of the neural networks. We demonstrate the viability and efficiency of the proposed neural networks in two specific paradigms for Nash-equilibrium seeking in a zero-sum game and distributed constrained nonlinear optimization.

Distributionally robust unsupervised domain adaptation

Obtaining ground-truth label information from real-world data along with uncertainty quantification can be difficult or even infeasible. In the absence of labeled data for a certain task, unsupervised domain adaptation (UDA) techniques have shown great accomplishment by learning transferable knowledge from labeled source domain data and applying it to unlabeled target domain data, yet scarce studies consider addressing uncertainties under domain shifts to improve the model robustness. Distributionally robust learning (DRL) is emerging as a high-potential technique for building reliable learning systems that are robust to distribution shifts. In this study, we propose a distributionally robust unsupervised domain adaptation (DRUDA) method to enhance the machine learning model generalization ability under input space perturbations. The DRL-based UDA learning scheme is formulated as a min–max optimization problem by optimizing worst-case perturbations of the training source data. Our Wasserstein distributionally robust framework can reduce the shifts in the joint distributions across domains. The proposed DRUDA has been tested on digit datasets and the Office-31 dataset, compared with other state-of-the-art domain adaptation techniques. Our experimental results show that the proposed DRUDA leads to improvements in domain adaptation accuracy performance on target domains.

Latency minimization for multiuser computation offloading in fog-radio access networks

Recently, the Fog-Radio Access Network (F-RAN) has gained considerable attention, because of its flexible architecture that allows rapid response to user requirements. In this paper, computational offloading in F-RAN is considered, where multiple User Equipments (UEs) offload their computational tasks to the F-RAN through fog nodes. Each UE can select one of the fog nodes to offload its task, and each fog node may serve multiple UEs. The tasks is computed by the fog nodes or further offloaded to the cloud via a capacity-limited fronhaul link. In order to compute all UEs' tasks quickly, joint optimization of UE-Fog association, radio and computation resources of F-RAN is proposed to minimize the maximum latency of all UEs. This min-max problem is formulated as a Mixed Integer Nonlinear Program (MINP). To tackle it, first, MINP is reformulated as a continuous optimization problem, and then the Majorization Minimization (MM) method is used to find a solution. The MM approach that we develop is unconventional in that each MM subproblem is solved inexactly with the same provable convergence guarantee as the exact MM, thereby reducing the complexity of MM iteration. In addition, a cooperative offloading model is considered, where the fog nodes compress-and-forward their received signals to the cloud. Under this model, a similar min-max latency optimization problem is formulated and tackled by the inexact MM. Simulation results show that the proposed algorithms outperform some offloading strategies, and that the cooperative offloading can exploit transmission diversity better than noncooperative offloading to achieve better latency performance.

Stability for Constrained Minimax Optimization

Stability for Constrained Minimax Optimization

Robust Beamforming Design for Dual-Function Radar-Communication System

In this paper, we investigate the problem of robust beamforming for multiple-input multiple-output (MIMO) dual-function radar-communication (DFRC) system, when there exist radar look direction mismatch and communication channel state information (CSI) error. To address these system imperfections, a region-of-interest (ROI) rather than a specific angle is considered for radar sensing, and the communication CSI is assumed to be available with a bounded unknown error. With these considerations, we design a robust dual-function beamformer through optimizing the radar sensing performance in terms of the energy radiated towards the ROI, while guaranteeing the communication quality-of-service (QoS) measured by the per-user signal-to-interference-plus-noise ratio (SINR) as required. For this robust design, a max-min optimization problem is formulated and a semidefinite programming (SDP) reformulation based on the S-procedure is presented. The popular semidefinite relaxation (SDR) technique is employed to solve the resulting non-convex problem, and the tightness of SDR is discussed. Further, we propose to incorporate an extra constraint of main beam control into the problem to achieve a flat magnitude response of the beampattern over the ROI. In addition, it is shown that the proposed approach is straightforwardly applicable to the scenario of multiple targets. Numerical examples are carried out to show the tightness, and demonstrate the effectiveness and robustness of the proposed MIMO-DFRC beamforming approach.

Robust energy management in isolated microgrids with hydrogen storage and demand response

Hydrogen is called to be one of the main energy vectors in future energy systems, especially for energy storage, where this carrier presents some interesting features. Its application in microgrids might help to improve economic, environmental and reliability indicators, providing greater storage capacity than other technologies, like batteries. Following this idea, this paper develops a robust energy management methodology for isolated microgrids considering hydrogen storage and demand response initiatives. The new proposal is raised as a nested max-min optimization framework. To reduce the original problem to a tractable single-level one, a master-slave scheme is developed by which binary variables can be fixed and thus the inner problem can be reduced to its equivalent Karush-Kuhn-Tucker conditions. Then, the overall optimization paradigm is solved in an iterative fashion used the Constraint-and-Column Generation Algorithm. The resulting tool is applied to a benchmark isolated microgrid, thus validating it for industry applications. Moreover, different relevant results are analysed in-depth. Thus, the role of hydrogen storage and demand response initiatives is discussed, concluding that flexible demand has a more notable impact on monetary savings than hydrogen storage, reducing the total cost by 6 % with respect to the base case. Some intrinsic issues are also identified. For example, it is observed that flexible consumers are more frequently called when the hydrogen chain is enabled, which may provoke undesirable effects like response fatigue.

Deep neural network classifier for multidimensional functional data

AbstractWe propose a new approach, called as functional deep neural network (FDNN), for classifying multidimensional functional data. Specifically, a deep neural network is trained based on the principal components of the training data which shall be used to predict the class label of a future data function. Unlike the popular functional discriminant analysis approaches which only work for one‐dimensional functional data, the proposed FDNN approach applies to general non‐Gaussian multidimensional functional data. Moreover, when the log density ratio possesses a locally connected functional modular structure, we show that FDNN achieves minimax optimality. The superiority of our approach is demonstrated through both simulated and real‐world datasets.

Feature optimization method for white feather broiler health monitoring technology

The existing research on white feather broiler health monitoring technology concentrate on the selection of classification algorithms and optimization of internal parameters, ignoring the white feather broiler sound signal data set (abbr. signal data set) and internal features for training classifiers. In this paper, the authors shifted the research focus to the signal data set, and proposed an applicable feature optimization method in order to improve the practicality and stability of broiler health monitoring technology. First, for outliers in the signal data set, the boxplot was used to detect them, and they were transformed into missing values. For all missing values in the signal data set, six missing value processing methods were used to process them, respectively. The optimal decision tree classifier (abbr. DT classifier) filling was determined. Second, three normalization methods were used to process the signal data set, respectively, and the optimal min–max normalization method was determined. Then, the Pearson correlation coefficient and the p-value were used to perform correlation tests on features and labels in the signal data set, then the linear–nonlinear-fusion-based (abbr. LNLF-based) feature selection method was newly proposed to select those features that have a great influence on the classification effect of the random forest classifier (abbr. RF classifier). Multiple tests had verified its superiority. Finally, ten white feather broiler sound signal validation data sets (abbr. validation data sets) were used to verify the proposed feature optimization method. It shows that the average classification accuracy achieved by the RF classifier on the signal data set before and after feature optimization improved from 72.85% to 85.28%, with an improvement of 12.43%. The average G-mean achieved by the RF classifier improved from 0.828 to 0.939, with an improvement of 0.111. The average classification accuracy and average G-mean achieved by the RF classifier on the validation data set before and after feature optimization improved significantly, with an average improvement of 13.26% and 0.093, respectively. This method improves the reliability of the existing classifiers in another way, which is an important supplement to the feature engineering study, and promotes the research progress of the existing white feather broiler health monitoring technology.

Resource allocation and BER performance analysis of NOMA based cooperative networks

Resource sharing and management can significantly improve the performance and spectral efficiency of wireless systems. In power domain non-orthogonal multiple access (NOMA) systems, users share a common bandwidth for simultaneous data transmission and therfore, the spectral efficiency of the system will be improved at the expense of increased complexity. Provided proper power allocation, it is possible to detect symbols at receivers. In this paper, the downlink of a cellular system is considered where a base station (BS) communicates with two users using NOMA with the assistance of a decode-and-forward (DF) relay. In the proposed scheme, receivers combine received signals from direct and cooperative links, and decode symbols by employing successive interference cancellation (SIC). The bit error rate (BER) performance of the proposed scheme is analyzed and the optimal power allocation is also derived through a Min–Max optimization, and a closed form approximate solution is proposed for binary phase shift keying (BPSK) modulation. Furthermore, it is proved that the proposed receiver achieves full diversity order for all users for proper choice of power allocation. Simulation results also corroborate BER and diversity analysis.

Robust Data-Enabled Predictive Control: Tractable Formulations and Performance Guarantees

We introduce a general framework for robust data-enabled predictive control (DeePC) for linear time-invariant (LTI) systems, which enables us to obtain robust and optimal control in a receding-horizon fashion based on inexact input and output data. Robust DeePC solves a min-max optimization problem to compute the optimal control sequence that is resilient to all possible realizations of the uncertainties in data within a prescribed uncertainty set. We present computationally tractable reformulations of the min-max problem with various uncertainty sets. Moreover, we show that even though an accurate prediction of the future behavior is unattainable due to inaccessibility of exact data, the obtained control sequence provides performance guarantees for the actually <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">realized</i> input and output cost in open loop. Finally, we demonstrate the performance of robust DeePC using high-fidelity simulations of a power converter system.

Learning-Based Event-Triggered Tracking Control for Nonlinear Networked Control Systems With Unmatched Disturbance

This article concentrates on optimal tracking control for a class of nonlinear networked systems subjecting to limited network bandwidth and unmatched disturbance. Given the models of the control and reference systems, the considered optimal tracking control issue is initially formulated as a minimax optimization problem. Then, with the introduction of an event-triggered mechanism used for saving bandwidth, the formulated problem is transformed into solving an event-based Hamilton-Jacobi-Isaacs (HJI) equation by recurring to the Bellman optimality theory. Based on the HJI equation, we demonstrate that the stability of the concerned system in the sense of uniformly ultimately bounded (UUB) can be guaranteed with the envisioned optimal control and worst disturbance policies. Here, the disturbance policy can be varied periodically while the control policy can only be updated at event-triggering instants, which differs from the existed researches. Furthermore, we propose a reinforcement learning (RL)-based algorithm to handle the constructed HJI equation and thus settle the studied tracking control problem. The effectiveness of the algorithm is finally validated by both theoretical analysis and simulations.

Distributed robust moving horizon estimation for multisensor systems with stochastic and norm‐bounded uncertainties

SummaryIn this study, a distributed robust moving horizon estimation (DRMHE) approach is proposed for time‐varying multisensor systems affected by stochastic and norm‐bounded uncertainties. First, by considering the uncertainties in all model matrices, stochastic min–max optimization problems in local and collective forms are presented to achieve the proposed estimator. Then, by converting the problems to robust regularized least‐squares problems with uncertain parameters, closed‐form solutions are obtained for estimations in both local and collective forms. Furthermore, the stability of the proposed estimator is investigated under some appropriate conditions. At last, two different examples are employed to show the robust performance and superiority of the proposed DRMHE approach compared to the existing methods.

Breaking the Sample Size Barrier in Model-Based Reinforcement Learning with a Generative Model

This paper studies a central issue in modern reinforcement learning, the sample efficiency, and makes progress toward solving an idealistic scenario that assumes access to a generative model or a simulator. Despite a large number of prior works tackling this problem, a complete picture of the trade-offs between sample complexity and statistical accuracy has yet to be determined. In particular, all prior results suffer from a severe sample size barrier in the sense that their claimed statistical guarantees hold only when the sample size exceeds some enormous threshold. The current paper overcomes this barrier and fully settles this problem; more specifically, we establish the minimax optimality of the model-based approach for any given target accuracy level. To the best of our knowledge, this work delivers the first minimax-optimal guarantees that accommodate the entire range of sample sizes (beyond which finding a meaningful policy is information theoretically infeasible).

Robust expansion planning and hardening strategy of meshed multi-energy distribution networks for resilience enhancement

As an integrated energy system (IES) incorporated with electricity, gas, and heating networks, the meshed multi-energy distribution network (MMDN) can operate more economically with higher energy utilization efficiency. To achieve high reliability, MMDN is mesh-constructed and radial-operated, complicating the operation scenarios involved in planning. Existing coordinated planning models seldom consider network reconfiguration and hardening strategies collaboratively for extreme contingencies, which is not consistent with actual conditions. In addition, considering the multi-stage discreteness of the expansion planning problem, the existing robust algorithms may feedback invalid cuts. The neglect of such issues could introduce a momentous impact on the optimality of solutions. This paper presents a resiliency-oriented expansion planning and hardening model of MMDN, and the hierarchical algorithm is developed. Specifically, the model is formulated as a min–max-min optimization problem including investment and operation levels. The investment level problem optimizes planning schemes constrained with the robustness verification of the operation level problem. The operation level problem examines the operational cost and feeds back iterative information to the investment level. An additional cuts generation method is developed in the operation level, which effectively deals with invalid feedback information caused by the existing binary variable parameterization. Furthermore, accelerating strategies including infeasible region reduction, multi-cut feedback, and contingency set reduction are developed. To be solved by off-the-shelf solvers, the model is cast as a mixed-integer linear programming (MILP) problem by adopting the convex hull relaxation and linearization techniques for operational constraints with high accuracy. Numerical test results justify the effectiveness of the model.

Optimal minimax rates of specification testing with data-driven bandwidth

This study investigates optimal minimax rates of specification testing for linear and non-linear instrumental variable regression models. The test constructed by non-parametric kernel techniques can be rate optimal when bandwidths are selected appropriately. Since bandwidths are often selected in a data-dependent way in empirical studies, the rate-optimality of the test with data-driven bandwidths is investigated. While least squares cross-validation selects bandwidths that are optimal for estimation, it is shown not to be optimal for testing. Thus, we propose a novel bandwidth selection method for testing, the performance of which is investigated in a simulation study.

Optimal estimation and computational limit of low-rank Gaussian mixtures

Structural matrix-variate observations routinely arise in diverse fields such as multilayer network analysis and brain image clustering. While data of this type have been extensively investigated with fruitful outcomes being delivered, the fundamental questions like its statistical optimality and computational limit are largely under-explored. In this paper, we propose a low-rank Gaussian mixture model (LrMM) assuming each matrix-valued observation has a planted low-rank structure. Minimax lower bounds for estimating the underlying low-rank matrix are established allowing a whole range of sample sizes and signal strength. Under a minimal condition on signal strength, referred to as the information-theoretical limit or statistical limit, we prove the minimax optimality of a maximum likelihood estimator which, in general, is computationally infeasible. If the signal is stronger than a certain threshold, called the computational limit, we design a computationally fast estimator based on spectral aggregation and demonstrate its minimax optimality. Moreover, when the signal strength is smaller than the computational limit, we provide evidences based on the low-degree likelihood ratio framework to claim that no polynomial-time algorithm can consistently recover the underlying low-rank matrix. Our results reveal multiple phase transitions in the minimax error rates and the statistical-to-computational gap. Numerical experiments confirm our theoretical findings. We further showcase the merit of our spectral aggregation method on the worldwide food trading dataset.

Optimal control of inventory level for perishable goods with uncertain decay factor and uncertain forecast information: a new robust MPC approach

This paper deals with the inventory control in supply chains under the following assumptions: (1) perishable goods with uncertain deteriorating factor, (2) a future uncertain customer demand that, over a limited prediction horizon, belongs to a known compact set. The problem is to define a smooth control policy maximising the fulfilled customer demand and minimising the inventory level. This problem is here solved through a new Robust Model Predictive Control (RMPC) approach. This implies solving a min–max optimisation problem with hard constraints on the control effort (i.e. the sequence of replenishment orders). To drastically reduce the numerical complexity of this problem, the control signal is sought in the space of B-spline functions, which are known to be universal approximators admitting a parsimonious parametric representation. This allows us: (1) to reduce the number of both decision variables and constraints involved in the optimisation procedure, (2) to reformulate the numerically demanding minimisation of the worst case cost functional as a simpler Weighted Constrained Robust Least Squares (WCRLS) estimation problem. The WCRLS algorithm can be efficiently solved using interior point methods. A rigorous analysis of stability and feasibility conditions is provided.