Non-convex Optimization Problem Research Articles

Local dual-graph discriminant classifier for binary classification

Graph-based methods mine the potential structural information of data by constructing various graphs that positively affect the classifiers when dealing with classification problems. However, traditional graph-based classifiers are the most common single-graph classifiers and minimize only intra-class compactness, where inter-class separability is replaced by other factors. To consider real inter-class separability, we introduce a novel local dual-graph structure that can fully mine the geometric distribution of data by simultaneously maximizing the inter-class separability and minimizing the intra-class compactness. This local dual-graph structure reflects the relationship between samples and their neighbors and hence avoids the negative impact of outliers on the construction of graphs. Furthermore, a novel classifier called the local dual-graph discriminant classifier (LDGDC) is proposed using a local dual-graph structure. Originally, LDGDC is designed to perform the following optimization: minimization of the 2-norm regularization of model coefficients and intra-class compactness, and maximization of the inter-class separability, which is a non-convex optimization problem. To facilitate the solution, we transform the original non-convex problem of LDGDC into a convex problem. Finally, experiments were conducted on several public datasets, and the results demonstrate the effectiveness and robustness of the proposed LDGDC.

Normalized penalty gradient flow: A continuous-time approach for solving constrained nonconvex nonsmooth optimization problems

To avoid calculating penalty parameters, this paper introduces a continuous-time approach that combines the normalized gradient flow with the penalty method to solve the nonconvex nonsmooth optimization problem with a convex constraint set. Subsequently, this approach is extended to solve nonconvex nonsmooth optimization with a box constraint set and affine equality constraints. Compared with the current methods, the proposed approach does not require calculating the penalty parameters, allows the objective function to be nonconvex or nonsmooth, and allows the initial point to be arbitrarily selected. It is proved that the solution trajectory with any initial point converges to the critical point set of the optimization problem. Finally, the effectiveness of the proposed approach is authenticated through several numerical experiments.

Evaluation method of distributed photovoltaic maximum access capacity based on ADMM

With the increasing penetration rate of distributed photovoltaics in the distribution network, new requirements are put forward for the operation and control of distribution networks. In order to ensure that the distribution network can operate stably within a safe range, it is important to evaluate the maximum capacity of the distribution network that can be connected to distributed photovoltaics. Considering the high cost of centralized computing, high communication requirements, and poor reliability, it is difficult to adapt to the actual needs of the distribution network with large-scale access to distributed photovoltaics. Therefore, this paper proposes an ADMM-based distributed PV maximum access capacity evaluation method. This method takes the distributed photovoltaic access capacity and the active power loss of the distribution network line as the objective function and considers the power flow equation constraints, node voltage constraints, and line current carrying capacity constraints of the distribution network. By simplifying the model, the non-convex optimization problem is transformed into a convex quadratic programming problem, and the ADMM algorithm is used to solve the distributed solution to obtain the access strategy of distributed PV in various regions under different operating conditions. Finally, the IEEE-33 system is used as an example to simulate the calculation, and the experimental results verify the correctness and effectiveness of the method.

Joint resource allocation and beamforming design for RIS-aided symbiotic radio networks: A DRL approach

In this paper, we investigate a Reconfigurable Intelligent Surface (RIS)-assisted secure Symbiosis Radio (SR) network to address the information leakage of the primary transmitter (PTx) to potential eavesdroppers. Specifically, the RIS serves as a secondary transmitter in the SR network to ensure the security of the communication between the PTx and the Primary Receiver (PRx), and simultaneously transmits its information to the PTx concurrently by configuring the phase shifts. Considering the presence of multiple eavesdroppers and uncertain channels in practical scenarios, we jointly optimize the active beamforming of PTx and the phase shifts of RIS to maximize the secrecy energy efficiency of RIS-supported SR networks while satisfying the quality of service requirement and the secure communication rate. To solve this complicated non-convex stochastic optimization problem, we propose a secure beamforming method based on Proximal Policy Optimization (PPO), which is an efficient deep reinforcement learning algorithm, to find the optimal beamforming strategy against eavesdroppers. Simulation results show that the proposed PPO-based method is able to achieve fast convergence and realize the secrecy energy efficiency gain by up to 22% when compared to the considered benchmarks.

Market design for ancillary service provisions of inertia and frequency response via virtual power plants: A non-convex bi-level optimisation approach

The Great Britain (GB) government is paving the path to decarbonisation by actively promoting the integration of wind power into its generation mix. This sharp transition to renewable energy, however, introduces specific challenges. The characteristics of non-synchronous wind turbines have the potential to impact grid stability and frequency security due to the reduction in system inertia. In response to these challenges, the deployment of virtual power plants (VPPs) is envisioned in deregulated power systems, which allow for the aggregation and coordinated control of diverse distributed energy resources, enhancing grid flexibility, reliability, and efficiency. Moreover, VPPs offer the provision of system inertia and frequency response services at the national level. This study drops this assumption and leverages the localised flexibility inherent in VPPs to meet the ancillary service requirements of the low-carbon power system. The formulated problem is constructed as a non-convex bi-level optimisation problem, where the upper-level problem represents the operation of a frequency-constrained unit commitment model, and the lower-level problem embodies the energy dispatches and frequency responses of a group of VPPs, guided by the dual price signals of cleared energy and ancillary services. To address the non-convex nature of the unit commitment problem, a two-fold approach is employed. First, binary commitment status decision variables are relaxed to continuous versions. Subsequently, the duality gap between the original non-convex unit commitment problem and its relaxed dual form is minimised, yielding a solution closely approximating the optimal solution of the original problem. Second, the relaxed bi-level optimisation problem is transformed into a single-level mathematical programs with equilibrium constraints by replacing the lower-level VPP problems with their equivalent Karush-Kuhn–Tucker optimality conditions. The case studies conducted in this work encompass a comprehensive scope. Initially, the study evaluates the effectiveness of VPPs in delivering ancillary services within the projected context of the GB power system for 2030. The findings reveal a substantial 28.28% decrease in system operation costs when compared to the benchmark case lacking VPP flexibility. Additionally, the integration of VPPs leads to significantly lower price signals for inertia, primary frequency response, and enhanced frequency response, reduced by 81.21%, 79.13%, and 86.27%, respectively, in contrast to the benchmark case. The analysis then delves into a sensitivity exploration, investigating the inherent flexibility of VPPs. The profit of local VPPs is expected to increase with VPP flexibility. Finally, the study illustrates how VPPs can adapt to and derive benefits from an increasing level of wind power penetration by providing a higher amount of frequency response.

D4R: Doubly robust reduced rank regression in high dimension

In this paper, we study high-dimensional reduced rank regression and propose a doubly robust procedure, called D4R, meaning concurrent robustness to both outliers in predictors and heavy-tailed random noise. The proposed method uses the composite gradient descent based algorithm to solve the nonconvex optimization problem resulting from combining Tukey’s biweight loss with spectral regularization. Both theoretical and numerical properties of D4R are investigated. We establish non-asymptotic estimation error bounds under both the Frobenius norm and the nuclear norm in the high-dimensional setting. Simulation studies and real example show that the performance of D4R is better than that of several existing estimation methods.

An Optimal Control Approach to the Problem of the Longest Self-Supporting Structure

The characterization of the self-supporting slender structure with the furthest length is of interest both from a mechanical and biological point of view. Indeed, from a mechanical perspective, this classical problem was developed and studied with different methods, for example using similarity solutions and stable manifolds. However, none of them led to a complete analytical solution. On the other hand, plant structures such as tree branches or searcher shoots in climbing plants can be considered elastic cantilevered beams. In this paper, we formulate the problem as a non-convex optimisation problem with mixed state constraints. The problem is solved by analysing the corresponding relaxation. With this method, it is possible to obtain an analytical characterization of the cross-section

Piecewise non-linear model predictive control with bounded disturbance

This work addresses the problem of robust [Formula: see text] model predictive control for constrained piecewise non-linear systems corrupted by norm-bounded disturbances for the first time. In this approach, the system can have different operating points with different subregions. In each subregion, the system is made up of an affine model disturbed by an additive non-linear term that is locally Lipschitz. The employment of the piecewise non-linear model leads to a non-convex optimization problem, which is far more difficult to solve. It is also much more challenging to develop the [Formula: see text] model predictive control for piecewise non-linear systems. The proposed method introduces a control strategy in the form of a convex optimization problem subject to linear matrix inequality constraints, with the ability to minimize the L2 gain between the disturbance input and the controlled output. The proposed controller guarantees system stability with a prescribed [Formula: see text] disturbance attenuation level under switching between subregions. Simulations on a highly non-linear chemical process are conducted to demonstrate the efficacy of the proposed approach.

Characterizing the nonemptiness and compactness of weakly efficient solution sets for non-convex multiobjective optimization problems

In this paper, we demonstrate that non-convex multiobjective optimization problems have nonempty and compact weakly efficient solution sets if and only if finite scalar optimization problems have nonempty and compact solution sets under mild conditions, which partially reduces the gap between non-convex and convex multiobjective problems in characterizing the nonemptiness and compactness of weakly efficient solution sets. Two examples are provided to support the findings.

Reliable Frequency Regulation Through Vehicle-to-Grid: Encoding Legislation with Robust Constraints

Problem definition: Vehicle-to-grid increases the low utilization rate of privately owned electric vehicles by making their batteries available to electricity grids. We formulate a robust optimization problem that maximizes a vehicle owner’s expected profit from selling primary frequency regulation to the grid and guarantees that market commitments are met at all times for all frequency deviation trajectories in a functional uncertainty set that encodes applicable legislation. Faithfully modeling the energy conversion losses during battery charging and discharging renders this optimization problem nonconvex. Methodology/results: By exploiting a total unimodularity property of the uncertainty set and an exact linear decision rule reformulation, we prove that this nonconvex robust optimization problem with functional uncertainties is equivalent to a tractable linear program. Through extensive numerical experiments using real-world data, we quantify the economic value of vehicle-to-grid and elucidate the financial incentives of vehicle owners, aggregators, equipment manufacturers, and regulators. Managerial implications: We find that the prevailing penalties for nondelivery of promised regulation power are too low to incentivize vehicle owners to honor the delivery guarantees given to grid operators. Funding: This work was supported by the Institut Vedecom. Supplemental Material: The online appendix is available at https://doi.org/10.1287/msom.2022.0154 .

On Weighted Sum Rate of Multi-User Photon-Counting Multiple-Input Multiple-Output Visible Light Communication Systems under Poisson Shot Noise

Photon counting has been proven to possess excellent signal detection capabilities at low power levels and has extensive potential applications in sixth-generation (6G) communications. However, the inherent dependency between the signal and noise complicates system analysis, and optimizing achievable rates in photon-counting visible light communication (VLC) systems remains unresolved. This paper introduces a new method aimed at minimizing multi-user interference (MUI) through a zero-forcing (ZF) scheme and maximizing the weighted sum rate of the proposed downlink multi-user photon-counting multiple-input multiple-output (MU-PhC-MIMO) VLC system by solving an optimization problem. The key point lies in our utilization of the ZF approach to derive a reasonable asymptotic approximation expression for the weighted sum rate. Subsequently, we use variable substitution and methods like successive convex approximation (SCA) to iteratively convexify the non-convex optimization problem and maximize the weighted sum rate under the ZF form. Compared to other algorithms, this approach can save 2.5 dB of transmission power to achieve the same system-weighted sum rate and significantly outperforms the repetition coding scheme at sufficient transmission power.

Multi-Agent-Deep-Reinforcement-Learning-Enabled Offloading Scheme for Energy Minimization in Vehicle-to-Everything Communication Systems

Offloading computation-intensive tasks to mobile edge computing (MEC) servers, such as road-side units (RSUs) and a base station (BS), can enhance the computation capacities of the vehicle-to-everything (V2X) communication system. In this work, we study an MEC-assisted multi-vehicle V2X communication system in which multi-antenna RSUs with liner receivers and a multi-antenna BS with a zero-forcing (ZF) receiver work as MEC servers jointly to offload the tasks of the vehicles. To control the energy consumption and ensure the delay requirement of the V2X communication system, an energy consumption minimization problem under a delay constraint is formulated. The multi-agent deep reinforcement learning (MADRL) algorithm is proposed to solve the non-convex energy optimization problem, which can train vehicles to select the beneficial server association, transmit power and offloading ratio intelligently according to the reward function related to the delay and energy consumption. The improved K-nearest neighbors (KNN) algorithm is proposed to assign vehicles to the specific RSU, which can reduce the action space dimensions and the complexity of the MADRL algorithm. Numerical simulation results show that the proposed scheme can decrease energy consumption while satisfying the delay constraint. When the RSUs adopt the indirect transmission mode and are equipped with matched-filter (MF) receivers, the proposed joint optimization scheme can decrease the energy consumption by 56.90% and 65.52% compared to the maximum transmit power and full offloading schemes, respectively. When the RSUs are equipped with ZF receivers, the proposed scheme can decrease the energy consumption by 36.8% compared to the MF receivers.

Optimal Innovation-Based Stealthy Attacks in Networked LQG Systems With Attack Cost.

In this article, the optimal innovation-based attack strategy is investigated for the networked linear quadratic Gaussian (LQG) systems. To bypass the detector, the attacks are required to follow strict stealthiness or ϵ -stealthiness described by the Kullback-Leibler divergence. The attackers aim to increase the quadratic control cost and decrease the attack cost, which is formulated as a nonconvex optimization problem. Then, based on the cyclic property of the matrix trace, the nonconvex objective function is transformed into a linear function related to attack matrices and covariance matrices of the tampered innovations. The optimal strictly stealthy attack is obtained by utilizing the matrix decomposition technique. Furthermore, the optimal ϵ -stealthy attack is derived to achieve a higher-attack effect by an integrated convex optimization, which distinguishes from the existing suboptimal attacks developed by a two-stage optimization. Simulation results are provided to show the effectiveness of the designed attacks.

Decentralized Caching Under Nonuniform File Popularity and Size: Memory-Rate Tradeoff Characterization

This paper aims to characterize the memory-rate tradeoff for decentralized caching under nonuniform file popularity and size. We consider a recently proposed decentralized modified coded caching scheme (D-MCCS) and formulate the cache placement optimization problem to minimize the average rate for the D-MCCS. To solve this challenging non-convex optimization problem, we first propose a successive Geometric Programming (GP) approximation algorithm, which guarantees convergence to a stationary point but has high computational complexity. Next, we develop a low-complexity file-group-based approach, where we propose a popularity-first and size-aware (PF-SA) cache placement strategy to partition files into two groups, taking into account the nonuniformity in file popularity and size. Both algorithms do not require the knowledge of active users beforehand for cache placement. Numerical results show that they perform very closely to each other. We further develop a lower bound for decentralized caching under nonuniform file popularity and size as a non-convex optimization problem and solved it using a similar successive GP approximation algorithm. We show that the D-MCCS with the optimized cache placement attains this lower bound when no more than two active users request files at a time. The same is true for files with uniform size but nonuniform popularity and the optimal cache placement being symmetric among files. In these cases, the optimized D-MCCS characterizes the exact memory-rate tradeoff for decentralized caching. For general cases, our numerical results show that the average rate achieved by the optimized D-MCCS is very close to the lower bound.

Control/physical systems co-design with spectral temporal logic specifications and its applications to MEMS

‘Co-design’ problems try to simultaneously design the physical and control components to improve the overall system performance. However, existing co-design paradigms cannot deal with complex frequency temporal domain specifications. In this paper, we investigate the co-design problem for a class of linear parameter-varying (LPV) systems with frequency temporal domain specifications. Firstly, the frequency temporal domain specifications are written in a formal language called spectral temporal logic (STL). Secondly, the satisfaction conditions of the spectral temporal logic specifications have been transformed into non-linear matrices inequality forms with necessary and sufficient conditions. Thirdly, the co-design problem is transformed into a non-convex optimisation problem with mixed-integer linear matrix inequalities (MILMIs) constraints, and then an iterative algorithm is proposed to solve the co-design problem with semidefinite programming (SDP). Finally, the performance of the algorithm and the expressiveness of spectral temporal logic are illustrated with the applications to micro-electromechanical systems (MEMS).

Asymmetric Supervised Fusion-Oriented Hashing for Cross-Modal Retrieval.

Hashing technologies have been widely applied for large-scale multimodal retrieval tasks owing to their excellent performance in search and storage tasks. Although some effective hashing methods have been proposed, it is still difficult to handle the intrinsic linkages that exist among different heterogeneous modalities. Moreover, optimizing the discrete constraint problem through a relaxation-based strategy results in a large quantization error and leads to a suboptimal solution. In this article, we present a novel asymmetric supervised fusion-oriented hashing method, named (ASFOH), which investigates three novel schemes to remedy the above issues. Specifically, we first explicitly formulate the problem as matrix decomposition into a common latent representation and a transformation matrix, combined with an adaptive weight scheme and nuclear norm minimization to ensure the information completeness of multimodal data. Then, we associate the common latent representation with the semantic label matrix, thereby increasing the discriminative capability of the model by constructing an asymmetric hash learning framework, thus, making the generated hash codes more compact. Finally, an efficient discrete optimization iterative algorithm based on nuclear norm minimization is proposed to decompose the nonconvex multivariate optimization problem into several subproblems with analytical solutions. Comprehensive experiments on the MIRFlirck, NUS-WIDE, and IARP-TC12 datasets testify that ASFOH outperforms the compared state-of-the-art approaches.

A control theoretic framework for adaptive gradient optimizers

Gradient-based optimization and control frameworks have been utilized in several applications. The learning rate parameter is typically chosen following a schedule or using methods such as line search to enhance the convergence rate. Recently, the machine learning community has developed methodologies for automated tuning of the learning rate, known as adaptive gradient methods. This paper develops a control theory-inspired framework for modeling adaptive gradient methods that solve non-convex optimization problems. We first model the adaptive gradient methods in a state–space framework, which allows us to present simpler convergence proofs of prominent adaptive optimizers, such as AdaGrad, Adam, and AdaBelief. The proposed framework is constructive because it allows synthesizing new adaptive optimizers. To illustrate this fact, we then utilize the transfer function paradigm from classical control to propose a new variant of Adam, coined AdamSSM, and prove its convergence. We add an appropriate pole-zero pair in the transfer function from squared gradients to the second moment estimate. Applications on benchmark machine learning tasks of image classification using CNN architectures and language modeling using LSTM architecture demonstrate that the AdamSSM algorithm improves the gap between generalization accuracy and faster convergence than the recent adaptive gradient methods.

Cardinality Constrained Portfolio Optimization via Alternating Direction Method of Multipliers.

Inspired by sparse learning, the Markowitz mean-variance model with a sparse regularization term is popularly used in sparse portfolio optimization. However, in penalty-based portfolio optimization algorithms, the cardinality level of the resultant portfolio relies on the choice of the regularization parameter. This brief formulates the mean-variance model as a cardinality ( l0 -norm) constrained nonconvex optimization problem, in which we can explicitly specify the number of assets in the portfolio. We then use the alternating direction method of multipliers (ADMMs) concept to develop an algorithm to solve the constrained nonconvex problem. Unlike some existing algorithms, the proposed algorithm can explicitly control the portfolio cardinality. In addition, the dynamic behavior of the proposed algorithm is derived. Numerical results on four real-world datasets demonstrate the superiority of our approach over several state-of-the-art algorithms.

Solving a Class of Nonsmooth Nonconvex Optimization Problems Via Proximal Alternating Linearization Scheme with Inexact Information

For optimization problems minimizing the sum of two nonconvex and nonsmooth functions, we propose an alternate linearization method with inexact data. In many practical optimization applications, only the inexact information of the function can be obtained. The core idea of this method is to add a quadratic function term to the nonconvex function(called local convexification of nonconvex function), and then to construct an approximate proximal point model. In each iteration, a series of iteration points are obtained by solving subproblems alternately. It can be proved that, in the sense of inexact oracles, these iteration points converge to the stable point of the original problem, and theoretically show that the algorithm has good convergent properties.

A novel stepsize for gradient descent method

We propose a novel adaptive stepsize for the gradient descent scheme to solve unconstrained nonlinear optimization problems. With the convex and smooth objective satisfying locally Lipschitz gradient we obtain the complexity O(1k) of f(xk)−f⁎ at most. By using the idea of the new stepsize, we propose another new algorithm based on the projected gradient for solving a class of nonconvex optimization problems over a closed convex set. The computational experiments show the efficiency of the new method.