Convex Formulation Research Articles

Multiple-Instance Learning from Pairwise Comparison Bags

Multiple-instance learning (MIL) is a significant weakly supervised learning problem, where the training data consists of bags containing multiple instances and bag-level labels. Most previous MIL research required fully labeled bags. However, collecting such data is challenging due to the labeling costs or privacy concerns. Fortunately, we can easily collect pairwise comparison information, indicating one bag is more likely to be positive than the other. Therefore, we investigate a novel MIL problem about learning a bag-level binary classifier only from pairwise comparison bags. To solve this problem, we display the data generation process and provide a baseline method to train an instance-level classifier based on unlabeled-unlabeled learning. To achieve better performance, we propose a convex formulation to train a bag-level classifier and give a generalization error bound. Comprehensive experiments show that both the baseline method and the convex formulation achieve satisfactory performance, while the convex formulation performs better. 1

Fast correntropy-based multi-view clustering with prototype graph factorization

As a consequence of the ability to incorporate information from different perspectives, multi-view clustering has gained significant attention. Nevertheless, 1) its high computational cost, particularly when processing large-scale and high-dimensional multi-view data, restricts its applications in practice; and 2) complex noise in real-world data also challenges the robustness of existing algorithms. To tackle the above challenges, we develop a fast correntropy-based multi-view clustering algorithm with prototype graph factorization (FCMCPF). FCMCPF first adopts prototype graphs to effectively mitigate the complexity associated with graph construction, thereby reducing it from a quadratic complexity to a linear one. Then, it decomposes these prototype graphs under the correntropy criterion to robustly find the cluster indicator matrix without any post-processing. To solve the non-convex and non-linear model, we devise a fast half-quadratic-based strategy to first convert it into a convex formulation and then swiftly complete the optimization via the matrix properties of orthogonality and trace. The extensive experiments conducted on noisy and real-world datasets illustrate that FCMCPF is highly efficient and robust compared to other advanced algorithms, with comparable or even superior clustering effectiveness.

On the Convex Formulations of Robust Markov Decision Processes

Robust Markov decision processes (MDPs) are used for applications of dynamic optimization in uncertain environments and have been studied extensively. Many of the main properties and algorithms of MDPs, such as value iteration and policy iteration, extend directly to RMDPs. Surprisingly, there is no known analog of the MDP convex optimization formulation for solving RMDPs. This work describes the first convex optimization formulation of RMDPs under the classical sa-rectangularity and s-rectangularity assumptions. By using entropic regularization and exponential change of variables, we derive a convex formulation with a number of variables and constraints polynomial in the number of states and actions, but with large coefficients in the constraints. We further simplify the formulation for RMDPs with polyhedral, ellipsoidal, or entropy-based uncertainty sets, showing that, in these cases, RMDPs can be reformulated as conic programs based on exponential cones, quadratic cones, and nonnegative orthants. Our work opens a new research direction for RMDPs and can serve as a first step toward obtaining a tractable convex formulation of RMDPs. Funding: The work in the paper was supported, in part, by NSF [Grants 2144601 and 1815275]; and Agence Nationale de la Recherche [Grant 11-LABX-0047].

Distribution and transmission coordinated dispatch under joint electricity and carbon day-ahead markets

Worldwide efforts to attain a more renewable-based energy matrix have led to the creation of carbon emission-related markets. Simultaneously, the increasing penetration, mostly of renewable-based sources, in distribution networks may allow distribution system operators (DSOs) to change their traditional participation in wholesale markets as passive buyers. Over the last few years, many authors have investigated and proposed optimization models for the strategic participation of DSOs in electricity markets. Nonetheless, proposals regarding the DSO as a strategic agent in both electricity and carbon-related markets are rare. In this paper, we propose a stochastic bilevel formulation to address this gap. The economic and operational constraints of each stakeholder are considered in the model. Additionally, the problem is written in a convex formulation for which finite convergence to optimality is guaranteed. The formulation was tested for a 14-node transmission system and a 34-node distribution system in which dispatchable and non-dispatchable energy sources, energy storage systems, and demand response units are installed. The results show the DSO’s capacity to place strategic bids in both market environments to maximize its profit, which features intentionally taking financial losses in the carbon allowance market to maximize the overall profit due to the income maximization in the electricity market. Finally, an additional case study with a 123-node distribution system shows the proposed method’s scalability.

Data-Driven Stochastic Optimal Control With Safety Constraints Using Linear Transfer Operators

We provide a data-driven framework for optimal control of a continuous-time control-affine stochastic dynamical system. The proposed framework relies on the linear operator theory involving Perron-Frobenius (P-F) and Koopman operators. Our first result involving the P-F operator provides a convex formulation to the optimal control problem in the dual space of densities. This convex formulation of the stochastic optimal control problem leads to an infinite-dimensional convex program. The finite-dimensional approximation of the convex program is obtained using a data-driven approximation of the linear operators. We provide rigorous error bounds and convergence rate for the data-driven approximation of the linear operators and comment on the convergence rate of the optimal control w.r.t. data size. Our second results demonstrate using the Koopman operator, dual to the P-F operator, for the stochastic optimal control design. We show that the Hamilton Jacobi Bellman (HJB) equation can be expressed using the Koopman operator. We provide an iterative procedure along the lines of a popular policy iteration algorithm based on the data-driven approximation of the Koopman operator for solving the HJB equation. The two formulations, namely the convex formulation involving the P-F operator and the Koopman-based formulation using the HJB equation, can be viewed as dual to each other where the duality follows due to the dual nature of the P-F and Koopman operators. Finally, we present examples to demonstrate the efficacy of the developed framework and verify the convergence rates for the operator and optimal control numerically.

Hybrid uncertainty approach for management of energy storage-embedded soft open points in distribution grids

Energy storage-embedded soft open point (ESOP) is an innovative electronic device that incorporates energy storage capabilities and is designed to interconnect two or three neighboring distribution feeders. Its primary functions include active power control, reactive power compensation, and grid voltage regulation. On the other hand, distribution grids have been equipped with other distributed energy resources (DERs) such as smart PV inverters, renewable energy sources (solar and wind), and so on. Therefore, this article proposes a novel management scheme for the mentioned devices in distribution grids based on a convex formulation to take powerful solvers like CPLEX into account. Since there are a set of uncertainties associated with the model, including renewable resources, loads, and market price, a hybrid framework is developed based on the unscented transformation (UT) method and information gap decision theory (IGDT). In other words, the uncertainties of renewable resources and loads are captured by the UT method which could provide a correlation among uncertain parameters, while the IGDT method is employed to model the uncertainty associated with the market price. The hybridization of these methods assists in reducing the discrepancy between the actual and predicted values of uncertain parameters. The model is applied to a 33-bus grid, considering various case studies. The proposed method results in a substantial operational cost reduction of approximately 26.82 %. Additionally, it ensures a flatter voltage profile across the network, with the minimum voltage reaching 0.985 p.u.

A comparison on power flow models for optimal power flow studies in integrated medium-low voltage unbalanced distribution systems

The uptake of distributed energy resources has led to new challenges in distribution system operation. These challenges include voltage and congestion issues along with a higher degree of unbalance due to the installation of mostly single-phase resources. This calls for more accurate distribution system modelling to perform three-phase power flow and optimal power flow analysis to ensure a safe operation. This paper provides a comparison of four unbalanced three-phase power flow formulations to be utilised in optimal power flow-based studies. To adapt these formulations to realistic-sized integrated medium and low voltage unbalanced distribution networks, we propose a linear model for low voltage transformers that is suitable to apply in convex power flow models. The accuracy of these models is compared to the non-linear model considering voltage and current magnitude errors, as well as voltage unbalance factor as performance indices. Our results show that out of the convex formulations, the linear approximated model, which takes the couplings between phases into account, combined with our transformer linearisation, is the most accurate at capturing voltage and current magnitudes. The voltage error of this model in our 5336-node and 3148-node medium-low voltage test networks remains less than 0.38% under a variety of loading and generation scenarios.

Multiple-instance Learning from Triplet Comparison Bags

Multiple-instance learning (MIL) solves the problem where training instances are grouped in bags, and a binary (positive or negative) label is provided for each bag. Most of the existing MIL studies need fully labeled bags for training an effective classifier, while it could be quite hard to collect such data in many real-world scenarios, due to the high cost of data labeling process. Fortunately, unlike fully labeled data, triplet comparison data can be collected in a more accurate and human-friendly way. Therefore, in this article, we for the first time investigate MIL from only triplet comparison bags , where a triplet (X a , X b , X c ) contains the weak supervision information that bag X a is more similar to X b than to X c . To solve this problem, we propose to train a bag-level classifier by the empirical risk minimization framework and theoretically provide a generalization error bound. We also show that a convex formulation can be obtained only when specific convex binary losses such as the square loss and the double hinge loss are used. Extensive experiments validate that our proposed method significantly outperforms other baselines.

An efficient equalization filter design approach for the accurate sound reproduction in a room environment with wide-band content

In some sound reproduction applications, a desired sound field needs to be reproduced by remote loudspeakers in a room with sufficient accuracy over a wide frequency band. When low-frequency room modes exist and the low-frequency content is required to be reproduced accurately, an equalization filter with a long time-domain response may be necessary, resulting in a filter with a significantly high filter order, especially when the sampling rate is high for the high-frequency content. This imposes computational challenges on both filter design and implementation. In this article, a convex formulation is proposed for the wide-band sound reproduction filter design. To address issues due to the low-frequency room modes, a reduced-order technique is then proposed based on a sub-band filter structure where a low sampling rate is used to equalize the low-frequency room mode characteristics, and a high sampling rate is used for high-frequency features which usually has a short time span in response. The computational time can be reduced by several orders of magnitude and the numerical stability can be improved when the proposed order reduction method is used to eliminate redundant orders. Simulation and experimental results confirm the accuracy and efficiency advantages of the proposed method.

Convex Formulation and Efficiency Enhancement for Powered Landing Guidance with Second-order Cone Probability Constraints

Convex Formulation and Efficiency Enhancement for Powered Landing Guidance with Second-order Cone Probability Constraints

MSGL+: Fast and Reliable Model Selection-Inspired Graph Metric Learning

The problem of learning graph-based data structures from data has attracted considerable attention in the past decade. Different types of data can be used to infer the graph structure, such as graphical Lasso, which is learned from multiple graph signals or graph metric learning based on node features. However, most existing methods that use node features to learn the graph face difficulties when the label signals of the data are incomplete. In particular, the pair-wise distance metric learning problem becomes intractable as the dimensionality of the node features increases. To address this challenge, we propose a novel method called MSGL+. MSGL+ is inspired from model selection, leverages recent advancements in graph spectral signal processing (GSP), and offers several key innovations: (1) Polynomial Interpretation: We use a polynomial function of a certain order on the graph Laplacian to represent the inverse covariance matrix of the graph nodes to rigorously formulate an optimization problem. (2) Convex Formulation: We formulate a convex optimization objective with a cone constraint that optimizes the coefficients of the polynomial, which makes our approach efficient. (3) Linear Constraints: We convert the cone constraint of the objective to a set of linear ones to further ensure the efficiency of our method. (4) Optimization Objective: We explore the properties of these linear constraints within the optimization objective, avoiding sub-optimal results by the removal of the box constraints on the optimization variables, and successfully further reduce the number of variables compared to our preliminary work, MSGL. (5) Efficient Solution: We solve the objective using the efficient linear-program-based Frank–Wolfe algorithm. Application examples, including binary classification, multi-class classification, binary image denoising, and time-series analysis, demonstrate that MSGL+ achieves competitive accuracy performance with a significant speed advantage compared to existing graphical Lasso and feature-based graph learning methods.

Multiple-Instance Learning From Unlabeled Bags With Pairwise Similarity

In <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">multiple-instance learning</i> (MIL), each training example is represented by a bag of instances. A training bag is either negative if it contains no positive instances or positive if it has at least one positive instance. Previous MIL methods generally assume that training bags are fully labeled. However, the exact labels of training examples may not be accessible, due to security, confidentiality, and privacy concerns. Fortunately, it could be easier for us to access the pairwise similarity between two bags (indicating whether two bags share the same label or not) and unlabeled bags, as we do not need to know the underlying label of each bag. In this paper, we provide the first attempt to investigate MIL from only similar-dissimilar-unlabeled bags. To solve this new MIL problem, we first propose a strong baseline method that trains an instance-level classifier by employing an unlabeled-unlabeled learning strategy. Then, we also propose to train a bag-level classifier based on a convex formulation and theoretically derive a generalization error bound for this method. Comprehensive experimental results show that our instance-level classifier works well, while our bag-level classifier even has better performance.

Efficient Allocation and Sizing the PV-STATCOMs in Electrical Distribution Grids Using Mixed-Integer Convex Approximation

Photovoltaic (PV) systems are a clean energy source that allows for power generation integration into electrical networks without destructive environmental effects. PV systems are usually integrated into electrical networks only to provide active power during the day, without taking full advantage of power electronics devices, which can compensate for the reactive power at any moment during their operation. These systems can also generate dynamic reactive power by means of voltage source converters, which are called PV-STATCOM devices. This paper presents a convex formulation for the optimal integration (placement and sizing) of PV-STATCOM devices in electrical distribution systems. The proposed model considers reducing the costs of the annual energy losses and installing PV-STATCOM devices. A convex formulation was obtained to transform the hyperbolic relation between the products of the voltage into a second-order constraint via relaxation. Two simulation cases in the two IEEE test systems (33- and 69-node) with radial and meshed topologies were implemented to demonstrate the effectiveness of the proposed mixed-integer convex model. The results show that PV-STATCOM devices reduce the annual cost of energy losses of electrical networks in a more significant proportion than PV systems alone.

Convex optimization of PV-battery system sizing and operation with non-linear loss models

To mitigate climate change, households are increasingly incentivized to install PV systems in combination with battery energy storage systems to increase their self-sufficiency and flexibility as well as to relieve the stress caused by the high penetration of distributed generation on the grid. This paper aims to assess the need for non-linear loss models as opposed to linear loss models found in the literature when optimizing the sizing and operation of PV-battery systems. Therefore, an optimization model is presented which implements non-linear, convex and linear converter and battery loss model formulations. The relaxed convex formulation is equivalent to the original non-linear formulation and can be solved more efficiently, decreasing the run time by a factor of 4. The impact of non-linear as opposed to linear loss models on the optimal solution is illustrated for a residential DC-coupled PV-battery system. The linear loss model is shown to result in an underestimation of the cost by 14.7% and the battery size by 12.4%. Further, the battery utilization is underestimated by a third. The proposed method is useful to accurately model the losses when optimizing the sizing and operation of a PV-battery system in exchange for a slightly higher computational time compared to linear loss models, though far below that of solving the non-relaxed non-linear problem.

A Convex Formulation of Frictional Contact Between Rigid and Deformable Bodies

A Convex Formulation of Frictional Contact Between Rigid and Deformable Bodies

Exact Excited-State Functionals of the Asymmetric Hubbard Dimer.

The exact functionals associated with the (singlet) ground state and the two singlet excited states of the asymmetric Hubbard dimer at half-filling are calculated using both Levy's constrained search and Lieb's convex formulation. While the ground-state functional is, as is commonly known, a convex function with respect to the density, the functional associated with the doubly excited state is found to be concave. Also, because the density-potential mapping associated with the first excited state is noninvertible, its "functional" is a partial, multivalued function composed of one concave and one convex branch that correspond to two separate domains of the external potential. Remarkably, it is found that, although the one-to-one mapping between density and external potential may not apply (as in the case of the first excited state), each state-specific energy and corresponding universal functional are "functions" whose derivatives are each other's inverse, just as in the ground state formalism.

Warmstarting strategies for convex optimization based multi-channel constrained active noise control filter design

In real-world applications of active noise control (ANC) systems, various constraints are supposed to be satisfied in the controller design process. The optimal control filter coefficients can be obtained by solving a constrained optimization problem, which usually requires a significant computational effort. Recently, a convex formulation in conic form was proposed for ANC filter design. The proposed formulation was shown to result in a computational time reduction by several orders of magnitude. It is of great interest to further improve its efficiency by using a priori information of the optimal filter coefficients. One potential way of achieving this goal is to introduce a warmstart technique so that the filter solution of a similar system or environment can be referred to for selecting the starting point of the optimization algorithm. However, the conic formulation should be solved by the interior-point method, which, in general, is challenging for applying warmstart techniques. In the current work, relaxation methods are proposed to the constraints of original ANC filter design formulation so that the warmstart techniques can be applied. Then, a warmstarting technique proposed in previous study is used to solve a series of perturbed problems, and the performance of the warmstarting technique is investigated. Results show that with the proposed modifications applied, the warmstarting strategy can significantly reduce the number of iterations needed for solving the conic formulation of the ANC filter design problem without much tuning efforts, and the method is effective and robust in various environmental setups.

Ellipsoidal Set-Based Extended Command Governor for Constrained Linear Systems With Bounded Disturbances

The purposes of this paper are twofold. The first is to derive a convex formulation for optimizing the fictitious dynamics of the extended command governor strategy recently proposed in [1]. The second purpose is to provide an efficient procedure to solve online the optimization problem of the ellipsoidal set based command governor or the extended command governor. Two numerical examples with comparison to earlier solutions from the literature illustrate the effectiveness of the proposed algorithm.

Convex Approach to Data-Driven Off-Road Navigation via Linear Transfer Operators

We consider the problem of optimal control design for navigation on off-road terrain. We use a traversability measure to characterize the difficulty of navigation on off-road terrain. The traversability measure captures terrain properties essential for navigation, such as elevation maps, roughness, slope, and texture. The terrain with the presence or absence of obstacles becomes a particular case of the proposed traversability measure. We provide a convex formulation to the off-road navigation problem by lifting the problem to the density space using the linear Perron-Frobenius (P-F) operator. The convex formulation leads to an infinite-dimensional optimal navigation problem for control synthesis. We construct the finite-dimensional approximation of the optimization problem using data. We use a computational framework based on the data-driven approximation of the Koopman operator. This makes the proposed approach data-driven and applicable to cases where an explicit system model is unavailable. Finally, we apply the proposed navigation framework with single integrator dynamics and Dubin's car model.

Bridging the Gap Between Artificial Neural Networks and Kernel Regressions for Vector-Valued Problems in Microwave Applications

Thanks to their convex formulation, kernel regressions have shown an improved accuracy with respect to artificial neural network (ANN) structures in regression problems where a reduced set of training samples are available. However, despite the above interesting features, kernel regressions are inherently less flexible than ANN structures since their implementations are usually limited to scalar-output regression problems. This article presents a vector-valued (multioutput) formulation of the kernel ridge regression (KRR) aimed at bridging the gap between multioutput ANN structures and scalar kernel-based approaches. The proposed vector-valued KRR relies on a generalized definition of the reproducing kernel Hilbert space (RKHS) and on a new multioutput kernel structure. The mathematical background of the proposed vector-valued formulation is extensively discussed together with different matrix kernel functions and training schemes. Moreover, a compression strategy based on the Nystrom approximation is presented to reduce the computational complexity of the model training. The effectiveness and the performance of the proposed vector-valued KRR are discussed on an illustrative example consisting of a high-speed link and on the optimization of a Doherty amplifier.