Non-convex Cost Research Articles

Convex modeling and control of multi-fuel compression ignition engines

Multi-fuel engines are complex machines that operate in varied ambient and operating conditions. For reliable operation of multi-fuel engines in varying operating conditions, proper modeling, and efficient control design for their combustion phasing are required. One of the practically implementable ways this is achieved in practice is by, Gaussian process regression (GPR) models and then designing a feedforward control through look-up tables (LUTs). The main approach of designing LUTs is based on the inversion of GPR models that can be formulated as an optimization problem. Although GPR models have good generalization properties and ensure accuracy with a limited number of data, their usage in optimization problems leads to non-convex costs which are computationally burdensome to solve. To address this problem, this work proposes alternative modeling approaches for multi-fuel engines based on non-parametric convex regression and convex neural networks. First, the experimental data collected from the 4-cylinder diesel engine is exploited to obtain convex models and then, the resulting convex models are used in the LUT design process. To demonstrate the success of the method, we compare the convex models and the GPR model in terms of accuracy, computational time in control design, and control performance, as well as present a comparison in terms of model complexity between the investigated model structures.

Multi-Objective Optimization for Load Balancing and Trading Scheduling in Networked Microgrids

Renewable energy resources are considered an integral part of networked microgrids. Networked microgrids provide an optimal solution for stable and reliable energy to the end consumers even in case of its islanded mode of operation. There are several explicit analytical mathematical formulations for optimization of operations of networked microgrids including load balancing and trade scheduling of energy. Mostly, the scheduling problem in networked microgrids is the job of distribution operators to use common scheduling tools. In this paper, we present the mathematical formulation of multi-objective cost functions and their optimization by using the multi-objective particle swarm optimization (MOPSO) algorithm. Multi-objective cost functions and duality gap are designed and then the problem is solved by introducing MOPSO which has more than one objective function. The total energy cost function is then optimized in order to get swarm swarm-optimal solution. This algorithm computed the local energy generation and demand of each networked microgrid and made the decision of what energy is needed to buy or sell supported by each iteration of the operation of networked microgrids. We used an optimization formulation and its dual conception in order to propose the problem as a multi-objective formulation. The proposed method can deal with convex and nonconvex cost functions. In the case of non-convex functions, we have used the uncertainty cost functions for renewable sources attached to the networked microgrids.

Sequential Mixed Cost-Based Multi-Sensor and Relative Dynamics Robust Fusion for Spacecraft Relative Navigation

The non-redescending convex functions degrade the filtering robustness, whereas the redescending non-convex functions improve filtering robustness, but they tend to converge towards local minima. This work investigates the properties of convex and non-convex cost functions from robustness and stability perspectives, respectively. To improve filtering robustness and stability to the high level of non-Gaussian noise, a sequential mixed convex and non-convex cost strategy is presented. To avoid the matrix singularity induced by applying the non-convex function, the M-estimation type Kalman filter is transformed into its information filtering form. Further, to address the problem of the estimation consistency in the iterated unscented Kalman filter, the iterated sigma point filtering framework is adopted using the statistical linear regression method. The simulation results show that, under different levels of heavy-tailed non-Gaussian noise, the mixed cost strategy can avoid the non-convex function-based filters falling into the local minimum, and further can improve the robustness of the convex function-based filter. Therefore, the mixed cost strategy provides a comprehensive improvement in the efficiency of the robust iterated filter.

Quantized Zeroth-Order Gradient Tracking Algorithm for Distributed Nonconvex Optimization Under Polyak-Łojasiewicz Condition.

This article focuses on distributed nonconvex optimization by exchanging information between agents to minimize the average of local nonconvex cost functions. The communication channel between agents is normally constrained by limited bandwidth, and the gradient information is typically unavailable. To overcome these limitations, we propose a quantized distributed zeroth-order algorithm, which integrates the deterministic gradient estimator, the standard uniform quantizer, and the distributed gradient tracking algorithm. We establish linear convergence to a global optimal point for the proposed algorithm by assuming Polyak-Łojasiewicz condition for the global cost function and smoothness condition for the local cost functions. Moreover, the proposed algorithm maintains linear convergence at low-data rates with a proper selection of algorithm parameters. Numerical simulations validate the theoretical results.

A reliable deterministic method for multi-area economic dispatch considering power losses

This paper offers a reliable deterministic method for solving multi-area economic dispatch (MAED). The proposed method can handle nonlinear power losses in electrical networks and discrete and nonlinear characteristics in power plants, such as fuel change facilities and nonconvex cost functions. To this end, the presented method reformulates the products of two variables in the power loss terms as one-dimensional expressions. Then, it converts all nonlinear terms in the MAED into linear segments. Moreover, it handles discrete decisions in fuel selection through binary variables. We assess the performance of the proposed method by some MAED test systems. The results show that the method can reliably obtain accurate solutions in the adopted test systems. It converges to costs of 10604.7, 654.7, 121592.7, 124628.4, and 169528.9 respectively in 2-area 4-unit, 3-area 10-unit, 4-area 40-unit, 2-area 40-unit, and 10-area 130-unit. These findings reproduce or outperform results found in earlier optimization algorithms. Moreover, the computational time evaluations indicate that the method can solve the adopted MAED problems in less than one minute, showing its efficiency. Overall, the obtained results confirm the efficacy of the method’s efficacy in handling the MAED with discrete and nonlinear terms.

Convex Predictor–Nonconvex Corrector Optimization Strategy with Application to Signal Decomposition

Many tasks in real life scenarios can be naturally formulated as nonconvex optimization problems. Unfortunately, to date, the iterative numerical methods to find even only the local minima of these nonconvex cost functions are extremely slow and strongly affected by the initialization chosen. We devise a predictor–corrector strategy that efficiently computes locally optimal solutions to these problems. An initialization-free convex minimization allows to predict a global good preliminary candidate, which is then corrected by solving a parameter-free nonconvex minimization. A simple algorithm, such as alternating direction method of multipliers works surprisingly well in producing good solutions. This strategy is applied to the challenging problem of decomposing a 1D signal into semantically distinct components mathematically identified by smooth, piecewise-constant, oscillatory structured and unstructured (noise) parts.

On fluorophore imaging by nonlinear diffusion model with dynamical iterative scheme

Fluorescence imaging aims at recovering the absorption coefficient of fluorophore in biological tissues. Due to the nonlinear dependence of excitation and emission fields on the unknown coefficient, such an inverse problem with the boundary measurement as inversion input is nonlinear. We reformulate this inverse problem as an optimization problem for a non-convex cost functional consisting of the unknown absorption coefficient depending both on the excitation field and the emission one with a penalty term. The existence of minimizer of the cost functional is proved rigorously, with explicit expression for the Fréchet derivative of the cost functional. For seeking the local minimizer considered as the approximate solution to the inverse problem efficiently, we develop a dynamical process which makes the non-convex cost functional quadratic at each iteration step, by freezing the unknown coefficient for the excitation field. This new dynamical quadratic optimization process is proven convergent and decreases the amount of computations for the fluorescence imaging, while the reconstruction accuracy keeps almost unchanged. Such advantages are verified by several numerical examples for different configurations of the absorption coefficient to be identified, comparing our proposed scheme with the algorithm for the original non-convex cost functional.

Generalized Nash equilibrium problems with mixed-integer variables

We consider generalized Nash equilibrium problems (GNEPs) with non-convex strategy spaces and non-convex cost functions. This general class of games includes the important case of games with mixed-integer variables for which only a few results are known in the literature. We present a new approach to characterize equilibria via a convexification technique using the Nikaido–Isoda function. To any given instance of the GNEP, we construct a set of convexified instances and show that a feasible strategy profile is an equilibrium for the original instance if and only if it is an equilibrium for any convexified instance and the convexified cost functions coincide with the initial ones. We develop this convexification approach along three dimensions: We first show that for quasi-linear models, where a convexified instance exists in which for fixed strategies of the opponent players, the cost function of every player is linear and the respective strategy space is polyhedral, the convexification reduces the GNEP to a standard (non-linear) optimization problem. Secondly, we derive two complete characterizations of those GNEPs for which the convexification leads to a jointly constrained or a jointly convex GNEP, respectively. These characterizations require new concepts related to the interplay of the convex hull operator applied to restricted subsets of feasible strategies and may be interesting on their own. Note that this characterization is also computationally relevant as jointly convex GNEPs have been extensively studied in the literature. Finally, we demonstrate the applicability of our results by presenting a numerical study regarding the computation of equilibria for three classes of GNEPs related to integral network flows and discrete market equilibria.

Feature extraction of small underwater sonar targets using neighborhood discriminant analysis

Linear discriminant analysis (LDA) stands as a widely used supervised feature extraction technique that maps data onto a low-dimensional subspace such that the between-class scatter is maximized while within-class scatter is minimized. Despite its utility, LDA faces many challenges, particularly when the within-class scatter matrix becomes singular due to small sample size. Other dimensionality reduction techniques, such as Neighbourhood Component Analysis (NCA), have been proposed as alternatives to LDA. NCA learns a linear transformation that maximizes the likelihood that datapoints of the same class are clustered together in the lower-dimensional space. However, the optimization of NCA relies on a non-convex cost function, making it prone to local minima. To address the challenges faced by both LDA and NCA, we propose a novel dimensionality reduction method named neighborhood discriminant analysis (NDA). Like NCA, NDA learns a linear transformation that aims to cluster datapoints based on class label. However, NDA is framed as an eigendecomposition problem, eliminating the need for non-convex optimization. We demonstrate the new approach on real small target sonar data. [Work funded by the ONR grant numbers N000142112420 and N000142312503, and DoD Navy (NEEC) Grant No. N001742010016.]

A Bayesian sampling optimisation strategy for finite element model updating

AbstractModel Updating (MU) aims to estimate the unknown properties of a physical system of interest from experimental observations. In Finite Element (FE) models, these unknowns are the elements’ parameters. Typically, besides model calibration purposes, MU and FEMU procedures are employed for the Non-Destructive Evaluation (NDE) and damage assessment of structures. In this framework, damage can be located and quantified by updating the parameters related to stiffness. However, these procedures require the minimisation of a cost function, defined according to the difference between the model and the experimental data. Sophisticated FE models can generate expensive and non-convex cost functions, which minimization is a non-trivial task. To deal with this challenging optimization problem, this work makes use of a Bayesian sampling optimisation technique. This approach consists of generating a statistical surrogate model of the underlying cost function (in this case, a Gaussian Process is used) and applying an acquisition function that drives the intelligent selection of the next sampling point, considering both exploitation and exploration needs. This results in a very efficient yet very powerful optimization technique, necessitating of minimal sampling volume. The performance of this proposed scheme is then compared to three well-established global optimisation algorithms. This investigation is performed on numerical and experimental case studies based on the famous Mirandola bell tower.

From NeurODEs to AutoencODEs: A mean-field control framework for width-varying neural networks

Abstract The connection between Residual Neural Networks (ResNets) and continuous-time control systems (known as NeurODEs) has led to a mathematical analysis of neural networks, which has provided interesting results of both theoretical and practical significance. However, by construction, NeurODEs have been limited to describing constant-width layers, making them unsuitable for modelling deep learning architectures with layers of variable width. In this paper, we propose a continuous-time Autoencoder, which we call AutoencODE, based on a modification of the controlled field that drives the dynamics. This adaptation enables the extension of the mean-field control framework originally devised for conventional NeurODEs. In this setting, we tackle the case of low Tikhonov regularisation, resulting in potentially non-convex cost landscapes. While the global results obtained for high Tikhonov regularisation may not hold globally, we show that many of them can be recovered in regions where the loss function is locally convex. Inspired by our theoretical findings, we develop a training method tailored to this specific type of Autoencoders with residual connections, and we validate our approach through numerical experiments conducted on various examples.

Two efficient logarithmic formulations to solve nonconvex economic dispatch

To solve economic dispatch (ED) with nonconvex nonlinear generation cost functions, this paper adopts a global optimization technique where piecewise linear approximation (PLA) functions replace the nonlinear functions. However, a problem with this technique is that the PLA model requires incorporating many discrete variables into the optimization model, which may render the model computationally expensive. This paper presents two efficient representation techniques of the PLAs, which scale logarithmically, including binary zig-zag (ZZB) and general integer zig-zag (ZZI). We replace the nonconvex cost functions in the ED with their PLAs and obtain an alternative ED formulation where two logarithmic techniques model the PLAs. Then, one can efficiently solve the alternative ED model with the standard branch-and-cut/bound-based algorithms to the desired optimality gap. To verify the techniques’ efficacy, we consider test systems having 13, 38, 40, 80, and 160 units. The numerical evaluations show that the logarithmic techniques presented can yield high-quality optimal solutions in less than 2 seconds in the considered cases, illustrating the techniques’ efficiency.

A Generalized Nesterov-Accelerated Second-Order Latent Factor Model for High-Dimensional and Incomplete Data.

High-dimensional and incomplete (HDI) data are frequently encountered in big date-related applications for describing restricted observed interactions among large node sets. How to perform accurate and efficient representation learning on such HDI data is a hot yet thorny issue. A latent factor (LF) model has proven to be efficient in addressing it. However, the objective function of an LF model is nonconvex. Commonly adopted first-order methods cannot approach its second-order stationary point, thereby resulting in accuracy loss. On the other hand, traditional second-order methods are impractical for LF models since they suffer from high computational costs due to the required operations on the objective's huge Hessian matrix. In order to address this issue, this study proposes a generalized Nesterov-accelerated second-order LF (GNSLF) model that integrates twofold conceptions: 1) acquiring proper second-order step efficiently by adopting a Hessian-vector algorithm and 2) embedding the second-order step into a generalized Nesterov's acceleration (GNA) method for speeding up its linear search process. The analysis focuses on the local convergence for GNSLF's nonconvex cost function instead of the global convergence has been taken; its local convergence properties have been provided with theoretical proofs. Experimental results on six HDI data cases demonstrate that GNSLF performs better than state-of-the-art LF models in accuracy for missing data estimation with high efficiency, i.e., a second-order model can be accelerated by incorporating GNA without accuracy loss.

Energy‐efficiency optimization and control for electric vehicle platooning with regenerating braking

AbstractIt is a critical problem to improve energy efficiency for electric vehicle platooning systems. Moreover, different from internal combustion engine vehicles, the electric engine has higher efficiency, and further regenerating braking is widely used to recycle part of the energy in the electric vehicle when it is braking. What is more, if vehicles take a formation to drive, they can save more energy. Combining all the favorable factors, this paper presents a two‐layer energy‐efficiency optimization strategy for electric vehicle platooning. The upper layer presents an optimization method to find the optimal velocities and distances between vehicles under different road conditions during the cruise status of the electric vehicle platooning. Due to the nonconvex cost function and considering regenerative braking, the optimization problem is addressed by the dynamic programming method combined with the successive convex approximation method. Further, the lower layer presents a real‐time Model Predictive Control (MPC) strategy, and it directly introduces the battery pack state of charge consumption as the input, which not only finishes the control mission but also consumes minimal energy. Finally, simulation results are provided to verify the effectiveness and advantages of the proposed methods.

The Nonlinear Kantorovich Transportation Problem with Nonconvex Costs

The Nonlinear Kantorovich Transportation Problem with Nonconvex Costs

Virtual reference feedback tuning with robustness constraints: A swarm intelligence solution

The simplified modeling of a complex system allied with a low-order controller structure can lead to poor closed-loop performance and robustness. A feasible solution is to avoid the necessity of a model by using data for the controller design. The Virtual Reference Feedback Tuning (VRFT) is a data-driven design method that only requires a single batch of data and solves a reference tracking problem, although with no guarantee of robustness. In this work, the inclusion of an H∞ robustness constraint to the VRFT cost function is addressed. The estimation of the H∞ norm of the sensitivity transfer function is extended to maintain the one-shot characteristic of the VRFT. Swarm intelligence algorithms are used to solve the non-convex cost function. The proposed method is applied in two real-world inspired problems with four different swarm intelligence algorithms, which are compared with each other through a Monte Carlo experiment of 50 executions. The obtained results are satisfactory, achieving the desired robustness criteria.

CACTO: Continuous Actor-Critic With Trajectory Optimization—Towards Global Optimality

This paper presents a novel algorithm for the continuous control of dynamical systems that combines Trajectory Optimization (TO) and Reinforcement Learning (RL) in a single framework. The motivations behind this algorithm are the two main limitations of TO and RL when applied to continuous nonlinear systems to minimize a non-convex cost function. Specifically, TO can get stuck in poor local minima when the search is not initialized close to a “good” minimum. On the other hand, when dealing with continuous state and control spaces, the RL training process may be excessively long and strongly dependent on the exploration strategy. Thus, our algorithm learns a “good” control policy via TO-guided RL policy search that, when used as initial guess provider for TO, makes the trajectory optimization process less prone to converge to poor local optima. Our method is validated on several reaching problems featuring non-convex obstacle avoidance with different dynamical systems, including a car model with 6D state, and a 3-joint planar manipulator. Our results show the great capabilities of CACTO in escaping local minima, while being more computationally efficient than the Deep Deterministic Policy Gradient (DDPG) and Proximal Policy Optimization (PPO) RL algorithms.

Fast electrical impedance tomography based on sparse Bayesian learning

Electrical impedance tomography (EIT) is a severely under-determined and ill-posed inverse problem. Therefore, based on the compressed sensing theory and the block sparse Bayesian learning (BSBL) model, a novel accelerated EIT sparse imaging algorithm is presented. The key feature of the proposed algorithm is that the convex–concave procedure (CCP) method is used to optimize the non-convex cost function of the block sparse Bayesian learning model which improves the convergence speed and reduces the time complexity of the algorithm. The proposed method can adaptively explore and utilize the inter-block sparsity and intra-block structural correlation of non-sparse signals without any a priori information, thereby sparse imaging is performed on non-sparse physiological data effectively. The results comparing with other methods show that the proposed algorithm not only reconstruct inclusions with different shapes and conductivity applicably and effectively but also reduce the running time of the algorithm.

Convexifying Market Clearing of SoC-Dependent Bids From Merchant Storage Participants

State-of-charge (SoC) dependent bidding allows merchant storage participants to incorporate SoC-dependent operation and opportunity costs in a bid-based market clearing process. However, such a bid results in a non-convex cost function in the multi-interval economic dispatch and market clearing, limiting its implementation in practice. We show that a simple restriction on the bidding format removes the non-convexity, making the multi-interval dispatch of SoC-dependent bids a standard convex piece-wise linear program.

Sublinear and Linear Convergence of Modified ADMM for Distributed Nonconvex Optimization

In this article, we consider distributed nonconvex optimization over an undirected connected network. Each agent can only access to its own local nonconvex cost function and all agents collaborate to minimize the sum of these functions by using local information exchange. We first propose a modified alternating direction method of multipliers (ADMM) algorithm. We show that the proposed algorithm converges to a stationary point with the sublinear rate <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {O}(1/T)$</tex-math></inline-formula> if each local cost function is smooth and the algorithm parameters are chosen appropriately. We also show that the proposed algorithm linearly converges to a global optimum under an additional condition that the global cost function satisfies the Polyak–Łojasiewicz condition, which is weaker than the commonly used conditions for showing linear convergence rates including strong convexity. We then propose a distributed linearized ADMM (L-ADMM) algorithm, derived from the modified ADMM algorithm, by linearizing the local cost function at each iteration. We show that the L-ADMM algorithm has the same convergence properties as the modified ADMM algorithm under the same conditions. Numerical simulations are included to verify the correctness and efficiency of the proposed algorithms.