Min-max Framework Research Articles

Regularized Simple Multiple Kernel k -Means With Kernel Average Alignment.

Multiple kernel clustering (MKC) aims to learn an optimal kernel to better serve for clustering from several precomputed basic kernels. Most MKC algorithms adhere to a common assumption that an optimal kernel is linearly combined by basic kernels. Based on a min-max framework, a newly proposed MKC method termed simple multiple kernel k -means (SimpleMKKM) can acquire a high-quality unified kernel. Although SimpleMKKM has achieved promising clustering performance, we observe that it cannot benefit from any prior knowledge. This would cause the learned partition matrix may seriously deviate from the expected one, especially in clustering tasks where the ground truth is absent during the learning course. To tackle this issue, we propose a novel algorithm termed regularized simple multiple kernel k -means with kernel average alignment (R-SMKKM-KAA). According to the experimental results of existing MKC algorithms, the average partition is a strong baseline to reflect true clustering. To gain knowledge from the average partition, we add the average alignment as a regularization term to prevent the learned unified partition from being far from the average partition. After that, we have designed an efficient solving algorithm to optimize the new resulting problem. In this way, both the incorporated prior knowledge and the combination of basic kernels are helpful to learn better unified partition. Consequently, the clustering performance can be significantly improved. Extensive experiments on nine common datasets have sufficiently demonstrated the effectiveness of incorporation of prior knowledge into SimpleMKKM.

Multiplicity‐1 minmax minimal hypersurfaces in manifolds with positive Ricci curvature

AbstractWe address the one‐parameter minmax construction for the Allen–Cahn energy that has recently lead to a new proof of the existence of a closed minimal hypersurface in an arbitrary compact Riemannian manifold with (Guaraco's work, relying on works by Hutchinson, Tonegawa, and Wickramasekera when sending the Allen–Cahn parameter to 0). We obtain the following result: if the Ricci curvature of N is positive then the minmax Allen–Cahn solutions concentrate around a multiplicity‐1 minimal hypersurface (possibly having a singular set of dimension ). This multiplicity result is new for (for it is also implied by the recent work by Chodosh–Mantoulidis). We exploit directly the minmax characterization of the solutions and the analytic simplicity of semilinear (elliptic and parabolic) theory in . While geometric in flavour, our argument takes advantage of the flexibility afforded by the analytic Allen–Cahn framework, where hypersurfaces are replaced by diffused interfaces; more precisely, they are replaced by sufficiently regular functions (from N to ), whose weighted level sets give rise to diffused interfaces. We capitalise on the fact that (unlike a hypersurface) a function can be deformed both in the domain N (deforming the level sets) and in the target (varying the values). We induce different geometric effects on the diffused interface by using these two types of deformations; this enables us to implement in a continuous way certain operations, whose analogues on a hypersurface would be discontinuous. An immediate corollary of the multiplicity‐1 conclusion is that every compact Riemannian manifold with and positive Ricci curvature admits a two‐sided closed minimal hypersurface, possibly with a singular set of dimension at most . (This geometric corollary also follows from results obtained by different ideas in an Almgren–Pitts minmax framework.)

Simple multiple kernel k-means with kernel weight regularization

Multiple kernel clustering (MKC) aims to determine the optimal kernel from several pre-computed basic kernels. Most of MKC algorithms follow a common assumption that the optimal kernel is linearly combined by basic kernels. Based on a min–max framework, a newly proposed MKC method termed simple multiple kernel k-means can acquire a high-quality unified kernel. Although it has achieved promising clustering performance, we have observed that it cannot benefit from any regularization or prior knowledge, thus the learned kernel weight coefficients may be seriously sparse or over-selected. To tackle this issue, we have developed a new algorithm termed simple multiple kernel k-means with kernel weight regularization (SMKKM-KWR), where we introduce average coefficients to avoid too sparse kernel weights. Specially, we add the average kernel coefficients as a regularization term to prevent the learned weight coefficients being far from the average values. After that, an efficient optimization strategy is proposed to solve the new resultant problem. By this way, the unified partition can learn clustering structure from fusion information of all the kernel views, with the goal to reach better clustering performance. Extensive experiments on nine benchmark datasets and four large-scale datasets have demonstrated the effectiveness and efficiency of the proposed algorithm.

Robustness Learning via Inference-Softmax Cross Entropy in Misaligned Distribution of Image

Adversarial examples easily mislead vision systems based on deep neural networks (DNNs) trained with softmax cross entropy (SCE) loss. The vulnerability of DNN comes from the fact that SCE drives DNNs to fit on the training examples, whereas the resultant feature distributions between the training and adversarial examples are unfortunately misaligned. Several state-of-the-art methods start from improving the inter-class separability of training examples by modifying loss functions, where we argue that the adversarial examples are ignored, thus resulting in a limited robustness to adversarial attacks. In this paper, we exploited the inference region, which inspired us to apply margin-like inference information to SCE, resulting in a novel inference-softmax cross entropy (I-SCE) loss, which is intuitively appealing and interpretable. The inference information guarantees that it is difficult for neural networks to cross the decision boundary under an adversarial attack, and guarantees both the inter-class separability and the improved generalization to adversarial examples, which was further demonstrated and proved under the min-max framework. Extensive experiments show that the DNN models trained with the proposed I-SCE loss achieve a superior performance and robustness over the state-of-the-arts under different prevalent adversarial attacks; for example, the accuracy of I-SCE is 63% higher than SCE under the PGD50un attack on the MNIST dataset. These experiments also show that the inference region can effectively solve the misaligned distribution.

A Comparative Study of Robust MPC and Stochastic MPC of Wind Power Generation System

In this paper, a complete comparison analysis of two advanced control algorithms, namely robust model predictive control (MPC) and stochastic MPC, is performed in order to optimize the operation of a wind power generation system (WPGS). The power maximization often conflicts with the mechanical load experienced by the turbine in the full-load region (i.e., the higher the power extracted, the higher the load) under the wind speed disturbance, thereby leading to high maintenance cost resulting from the fatigue damage. Thus, a typical 5 MW wind turbine operating in a high-speed region is considered to guarantee system security and economy. The robust MPC is designed by utilizing the min–max framework to track steady-state optimum operating reference trajectory with the deterministic constraint of output power, while the stochastic MPC is constructed by incorporating the invariant set theory to also ensure the system security subjecting to the probabilistic constraint of output power. The relation between the constraints and the implications on optimal performance are also studied. Comprehensive simulations on a mechanism model and FAST simulator are carried out to demonstrate the validation of the two control methods under various scenarios. It is discovered that when wind speed in the near future can be predicted and utilized in controller design, the stochastic MPC can effectively reduce the maintenance cost by suppressing the constraint violation rate compared to robust MPC with a similar energy utilization due to the incorporation of the stochastic characteristics of wind speed.

Acceleration of Deep Convolutional Neural Networks Using Adaptive Filter Pruning

While convolutional neural networks (CNNs) have achieved remarkable performance on various supervised and unsupervised learning tasks, they typically consist of a massive number of parameters. This results in significant memory requirements as well as a computational burden. Consequently, there is a growing need for filter-level pruning approaches for compressing CNN based models that not only reduce the total number of parameters but reduce the overall computation as well. We present a new min-max framework for the filter-level pruning of CNNs. Our framework jointly prunes and fine-tunes CNN model parameters, with an adaptive pruning rate, while maintaining the model's predictive performance. Our framework consists of two modules: (1) An adaptive filter pruning (AFP) module, which minimizes the number of filters in the model; and (2) A pruning rate controller (PRC) module, which maximizes the accuracy during pruning. In addition, we also introduce orthogonality regularization in training of CNNs to reduce redundancy across filters of a particular layer. In the proposed approach, we prune the least important filters and, at the same time, reduce the redundancy level in the model by using orthogonality constraints during training. Moreover, unlike most previous approaches, our approach allows directly specifying the desired error tolerance instead of the pruning level. We perform extensive experiments for object classification (LeNet, VGG, MobileNet, and ResNet) and object detection (SSD, and Faster-RCNN) over benchmarked datasets such as MNIST, CIFAR, GTSDB, ImageNet, and MS-COCO. We also present several ablation studies to validate the proposed approach. Our compressed models can be deployed at run-time, without requiring any special libraries or hardware. Our approach reduces the number of parameters of VGG-16 by an impressive factor of 17.5X, and the number of FLOPS by 6.43X, with no loss of accuracy, significantly outperforming other state-of-the-art filter pruning methods.

The amplification of the New Keynesian models and robust optimal monetary policy

This paper analyzes whether and how model uncertainty affects the amplification mechanism of the New Keynesian models in a simple min-max framework where the central bank plays a zero-sum game versus a hypothetical, evil agent. A first finding on a benchmark model with staggered price setting is that a robust optimal commitment policy necessitates more aggressive policy under a demand shock. Further, bringing additional persistence into the model deteriorates the effectiveness of monetary policy. Hence, allowing for either habit formation or partial indexation of prices to lagged inflation rate requires a stronger response for the policy to a demand shock. Together with the specification doubts, in order to reassure the private sector and signal that it will stabilize the fluctuations in the output gap, the policymaker reacts more aggressively as persistence rises. Although inflation persistence does not change the impact of model uncertainty, habit formation in consumption eliminates even reverses the impact of uncertainty on the policy reaction to a supply shock. In all cases, policymaker attributes less importance to nominal interest rate inertia with concerns about model uncertainty.

Adversarial Binary Collaborative Filtering for Implicit Feedback

Fast item recommendation based on implicit feedback is vital in practical scenarios due to data-abundance, but challenging because of the lack of negative samples and the large number of recommended items. Recent adversarial methods unifying generative and discriminative models are promising, since the generative model, as a negative sampler, gradually improves as iteration continues. However, binary-valued generative model is still unexplored within the min-max framework, but important for accelerating item recommendation. Optimizing binary-valued models is difficult due to non-smooth and nondifferentiable. To this end, we propose two novel methods to relax the binarization based on the error function and Gumbel trick so that the generative model can be optimized by many popular solvers, such as SGD and ADMM. The binary-valued generative model is then evaluated within the min-max framework on four real-world datasets and shown its superiority to competing hashing-based recommendation algorithms. In addition, our proposed framework can approximate discrete variables precisely and be applied to solve other discrete optimization problems.

Discrete-time minmax filtering subject to a norm-constrained state estimate

This paper considers discrete-time minmax filtering when the state estimate, or part of the state estimate, is subjected to a norm constraint. Derivation and use of minmax filters are motivated by the desire to estimate systems’ states in a more robust manner. There are various methods for enforcing constraints on the state estimate using the Kalman filter, however, enforcing constraints on the state estimate in the minmax framework has been less explored. This paper presents the derivation of a minmax filter for a discrete-time system with a full or partial equality norm constraint on the state estimate. This is accomplished by modifying the minmax filter’s optimization problem using Lagrange multipliers to enforce the constraint on the state estimates.

Learning classifiers from dual annotation ambiguity via a min–max framework

Many pattern recognition problems confront two sources of annotation ambiguity where (1) multiple annotators have provided their versions of a class label which may not be consistent with one another, which forms multi-labeler learning; (2) and meanwhile a class label is associated with a bag of input vectors or instances rather than each individual instance and a bag is positive for a class label as long as one of its instances shows an evidence of that class, which is often referred to as multi-instance learning. Existing methods for multi-labeler learning and multi-instance learning only address one source of the labeling ambiguity. They are not trivially feasible to tackle the dual ambiguity problem. We hence propose a novel optimization framework by modifying the hinge loss to employ the weighted consensus of different labelers׳ labels and further generalizing the notion of loss functions to bags of multiple instances. The proposed formulation can be approximately solved by two mathematically tractable models that accommodate two types of labeling bias. An alternating optimization algorithm has been derived to efficiently solve the two models. The proposed algorithms outperform existing methods on benchmark data sets collected for document classification, real-life crowd-sourced data sets, and a medical problem of heart wall motion analysis with diagnoses from multiple radiologists.

A Novel Learning Formulation in a unified Min-Max Framework for Computer Aided Diagnosis

The computer aided diagnosis (CAD) problems of detecting potentially diseased structures from medical images are typically distinguished by the following challenging characteristics: extremely unbalanced data between negative and positive classes; stringent real-time requirement of on- line execution; multiple positive candidates generated for the same malignant structure that are highly correlated and spatially close to each other. To address all these problems, we propose a novel learning formulation to combine cascade classification and multiple instance learning (MIL) in a unified min-max framework, leading to a joint optimization problem which can be converted to a tractable quadratic ally constrained quadratic program and efficiently solved by block-coordinate optimization algorithms. We apply the proposed approach to the CAD problems of detecting pulmonary embolism and colon cancer from computed tomography images. Experimental results show that our approach significantly reduces the computational cost while yielding comparable detection accuracy to the current state-of-the-art MIL or cascaded classifiers. Although not specifically designed for balanced MIL problems, the pro- posed method achieves superior performance on balanced MIL benchmark data such as MUSK and image data sets

Robust linear least squares regression

We consider the problem of robustly predicting as well as the best linear combination of d given functions in least squares regression, and variants of this problem including constraints on the parameters of the linear combination. For the ridge estimator and the ordinary least squares estimator, and their variants, we provide new risk bounds of order d/n without logarithmic factor unlike some standard results, where n is the size of the training data. We also provide a new estimator with better deviations in the presence of heavy-tailed noise. It is based on truncating differences of losses in a min–max framework and satisfies a d/n risk bound both in expectation and in deviations. The key common surprising factor of these results is the absence of exponential moment condition on the output distribution while achieving exponential deviations. All risk bounds are obtained through a PAC-Bayesian analysis on truncated differences of losses. Experimental results strongly back up our truncated min–max estimator.

A unified granular fuzzy-neuro min-max relational framework for medical diagnosis

We propose to accommodate herein our novel unified granular framework that uses a developed hybrid fuzzy-neuro relational system in order to tackle a complex medical diagnosis problem and to understand the influence of syndromes in relation to symptoms. To this goal, we propose to adapt our novel computational granular unified framework that is cognitively-motivated for learning IF-THEN fuzzy weighted diagnosis rules by using a hybrid neuro-fuzzy or fuzzy-neuro possibilistic model appropriately crafted as a means to automatically extract or learn diagnosis rules from only input-output examples by integrating some useful concepts from the human cognitive processes and adding some interesting granular functionalities. This learning scheme uses an exhaustive search over the fuzzy partitions of involved variables, automatic fuzzy hypotheses generation, formulation and testing, and approximation procedure of min-max relational equations. The main idea is to start learning from coarse fuzzy partitions of the involved proteins variations input variables and proceed progressively toward fine-grained partitions until finding the appropriate partitions that fit the data. According to the complexity of the problem at hand, it learns the whole structure of the fuzzy system, i.e., conjointly appropriate fuzzy partitions, appropriate fuzzy diagnosis rules, their number and their associated trapezoidal membership functions.

Min-Max Fair Power Flow Tracing for Transmission System Usage Cost Allocation: A Large System Perspective

Power flow tracing has been suggested as an approach for evaluating 1) transmission system usage (TSU) cost and 2) loss (MW) cost for generator and load entities in the system. Recently, optimal power flow tracing methods have been proposed to “explicitly” model fairness constraints in the tracing framework. This paper, further, strengthens the tracing-compliant min-max fair cost allocation approach. The min-max model proposed in this paper is robust. It addresses concerns like scalability, numerical stability and termination in a finite number of steps while searching the optimal solution. We also propose a methodology to model DISCOMs and GENCOs as coalition within min-max framework. Case studies on an all India network of 1699 nodes and comparison with average participation and marginal participation methods bring out the better conflict resolution feature of the proposed approach. A method to model HVDC lines within the marginal participation scheme is also proposed. Quantitative and qualitative comparison of various TSU cost allocation methods on such a large system is another noteworthy contribution of the paper.

Optimal Monetary Policy with an Uncertain Cost Channel

The cost channel of monetary transmission describes a supply‐side effect of interest rates on firms' costs. Previous research has found this effect to vary, both over time and across countries. Moreover, the cyclical nature of financial frictions is likely to amplify the cost channel. This paper derives optimal monetary policy in the presence of uncertainty about the true size of the cost channel. In a min–max approach, the central bank derives an optimal policy plan to be implemented by a Taylor rule. It is shown that uncertainty about the cost channel leads to an attenuated interest rate setting behavior. In this respect, the Brainard (1967) principle of cautious policy in the face of uncertainty continues to hold in both a Bayesian and a min–max framework.

Dynamic mean-variance portfolio analysis under model risk

The classical Markowitz approach to portfolio selection is compromised by two major shortcomings. First, there is considerable model risk with respect to the distribution of asset returns. Particularly, mean returns are notoriously difficult to estimate. Moreover, the Markowitz approach is static in that it does not account for the possibility of portfolio rebalancing within the investment horizon. We propose a robust dynamic portfolio optimization model to overcome both shortcomings. The model arises from an infinite-dimensional min-max framework. The objective is to minimize the worst-case portfolio variance over a family of dynamic investment strategies subject to a return target constraint. The worst-case variance is evaluated with respect to a set of conceivable return distributions. We develop a quantitative approach to approximate this intractable problem by a tractable one and report on numerical experiments.

Semisupervised SVM batch mode active learning with applications to image retrieval

Support vector machine (SVM) active learning is one popular and successful technique for relevance feedback in content-based image retrieval (CBIR). Despite the success, conventional SVM active learning has two main drawbacks. First, the performance of SVM is usually limited by the number of labeled examples. It often suffers a poor performance for the small-sized labeled examples, which is the case in relevance feedback. Second, conventional approaches do not take into account the redundancy among examples, and could select multiple examples that are similar (or even identical). In this work, we propose a novel scheme for explicitly addressing the drawbacks. It first learns a kernel function from a mixture of labeled and unlabeled data, and therefore alleviates the problem of small-sized training data. The kernel will then be used for a batch mode active learning method to identify the most informative and diverse examples via a min-max framework. Two novel algorithms are proposed to solve the related combinatorial optimization: the first approach approximates the problem into a quadratic program, and the second solves the combinatorial optimization approximately by a greedy algorithm that exploits the merits of submodular functions. Extensive experiments with image retrieval using both natural photo images and medical images show that the proposed algorithms are significantly more effective than the state-of-the-art approaches. A demo is available at http://msm.cais.ntu.edu.sg/LSCBIR/.

A new robust optimization approach for scheduling under uncertainty: II. Uncertainty with known probability distribution

In this work, we consider the problem of scheduling under uncertainty where the uncertain problem parameters can be described by a known probability distribution function. A novel robust optimization methodology, originally proposed by Lin, Janak, and Floudas [Lin, X., Janak, S. L., & Floudas, C. A. (2004). A new robust optimization approach for scheduling under uncertainty: I. Bounded uncertainty. Computers and Chemical Engineering, 28, 1069–1085], is extended in order to consider uncertainty described by a known probability distribution. This robust optimization formulation is based on a min–max framework and when applied to mixed-integer linear programming (MILP) problems, produces “robust” solutions that are immune against data uncertainty. Uncertainty is considered in the coefficients of the objective function, as well as the coefficients and right-hand-side parameters of the inequality constraints in MILP problems. Robust optimization techniques are developed for uncertain data described by several known distributions including a uniform distribution, a normal distribution, the difference of two normal distributions, a general discrete distribution, a binomial distribution, and a poisson distribution. The robust optimization formulation introduces a small number of auxiliary variables and additional constraints into the original MILP problem, generating a deterministic robust counterpart problem which provides the optimal/feasible solution given the (relative) magnitude of the uncertain data, a feasibility tolerance, and a reliability level. The robust optimization approach is then applied to the problem of short-term scheduling under uncertainty. Using the continuous-time model for short-term scheduling developed by Floudas and co-workers [Ierapetritou, M. G. & Floudas, C. A. (1998a). Effective continuous-time formulation for short-term scheduling: 1. Multipurpose batch processes. Ind. Eng. Chem. Res., 37, 4341–4359; Lin, X. & Floudas, C. A. (2001). Design, synthesis and scheduling of multipurpose batch plants via an effective continuous-time formulation. Comp. Chem. Engng., 25, 665–674], three of the most common sources of uncertainty in scheduling problems are explored including processing times of tasks, market demands for products, and prices of products and raw materials. Computational results on several examples and an industrial case study are presented to demonstrate the effectiveness of the proposed approach.