Nonconvex Terms Research Articles

Portfolio Construction as Linearly Constrained Separable Optimization

Mean-variance portfolio optimization problems often involve separable nonconvex terms, including penalties on capital gains, integer share constraints, and minimum position and trade sizes. We propose a heuristic algorithm for this problem based on the alternating direction method of multipliers (ADMM). This method allows for solve times in tens to hundreds of milliseconds with around 1000 securities and 100 risk factors. We also obtain a bound on the achievable performance. Our heuristic and bound are both derived from similar results for other optimization problems with a separable objective and affine equality constraints. We discuss a concrete implementation in the case where the separable terms in the objective are piecewise-quadratic, and we demonstrate their effectiveness empirically in realistic tax-aware portfolio construction problems.

On Image Segmentation Based on Local Entropy Fitting Under Nonconvex Regularization Term Constraints

The energy functional of the CV and LBF model is single, which makes the curve to get into the local minimum easily during the evolution process, and results inaccurate segmentation of the images with nonuniform grayscale and nonsmooth edges. The proposed algorithm, which is based on local entropy fitting under the constraint of nonconvex regularization term, is used to deal with such problems. In this algorithm, global information and local entropy are fitted to avoid segmentation falling into local optimum, and nonconvex regularization term is imported for constraint to protect edge smoothing. First, global information is used to evolve the approximate contour curve of the target segmentation. Then, a local energy functional with local entropy information is constructed to avoid the segmentation process from falling into a local minimum, and to precisely segment the image. Finally, nonconvex regularization terms are used in the energy functional to protect the smoothness of edge information during image segmentation process. The experimental results clearly indicate that the new algorithm can effectively resist noise, precisely segment images with nonuniform grayscale, and achieve the global optimal.

Enhanced total generalized variation method based on moreau envelope

Image restoration with total generalized variation (TGV) regularization has been proved to be a creditable method and used broadly, but this method is imperfect in retaining image details. This paper proposes a non-convex and non-separable regularization term based on TGV enhancement, which can maintain the strict convexity of the total cost function and avoid the underestimation of the TGV method. The new regularization is obtained by subtracting the Moreau envelope of TGV from the TGV term, in which the former is defined by infimal convolution. To minimize the cost function, the proximal forward-backward splitting (FBS) algorithm is applied, and the alternating direction method of multipliers with adaptive parameter estimation (APE-ADMM), which avoids the drawbacks of FBS algorithm, is proposed by applying the variable splitting and the augment Lagrangian method (ALM). This proposed algorithm can update the regularization parameter adaptively. Experiments varify that the proposed method is more accurate in solution estimation than the other promoted ones, and the new ADMM algorithm with new regularization shows improved effects of image restoration.

Combined higher order non-convex total variation with overlapping group sparsity for impulse noise removal

A typical approach to eliminate impulse noise is to use the l1-norm for both the data fidelity term and the regularization terms. However, the l1-norm tends to over penalize signal entries which is one of its underpinnings. Hence, we propose a variational model that uses the non-convex lp-norm, 0 < p < 1 for both the data fidelity and a second-order total variation regularization term combined with an overlapping group sparse regularizer. Specifically, to robustly eliminate impulse noise, the proposed method uses a non-convex data fidelity term. The hybrid combination of a second-order non-convex total variation and an overlapping group sparse regularization term is used to eliminate the remaining staircase artifacts while maintaining a sharp restored image. A mathematical formulation is derived and to implement it, the iterative re-weighted l1 (IRL1) based alternating direction method of multipliers (ADMM) is used to solve the constraints and the subproblems. Experimental results for image denoising and deblurring on several widely used standard images demonstrate that the proposed method performed better when compared to the l1-norm total variation (TV), total generalized variation (TGV) model, and the non-convex lp-norm TV-based data fidelity model in terms of peak signal-to-noise ratio (PSNR) and structure similarity index measure (SSIM).

A novel image denoising approach based on a non-convex constrained PDE: application to ultrasound images

In this paper, we are interested in the mathematical and simulation study of a new non-convex constrained PDE to remove the mixture of Gaussian–impulse noise densities. The model incorporates a non-convex data-fidelity term with a fractional constrained PDE. In addition, we adopt a non-smooth primal-dual algorithm to resolve the obtained proximal linearized minimization problem. The non-convex fidelity term is used to handle the high-frequency of the impulse noise component, while the fractional operator enables the efficient denoising of smooth areas, avoiding also the staircasing effect that appears on the relevant variational denoising models. Moreover, the proposed primal-dual algorithm helps in preserving fine structures and texture with good convergence rate. Numerical experiments, including ultrasound images, show that the proposed non-convex constrained PDE produces better denoising results compared to the state-of-the-art denoising models.

An efficient non-convex total variation approach for image deblurring and denoising

Total variation (TV) is broadly utilized in image processing because it is able to preserve sharp edges and object boundaries, which are usually the most important parts of an image. Recently, the non-convex functions such as the smoothly clipped absolute deviation, the minimax concave penalty, the capped ℓ1-norm penalty and the ℓp quasi-norm with 0<p<1 have been shown remarkable advantages in sparse learning due to the fact that they can overcome the over-penalization associated with the ℓ1-norm. In this paper, an efficient non-convex total variation approach for image deblurring and denoising model has been proposed, which combines a non-convex regularization term and a non-convex data fitting term perfectly. Firstly, the non-convex functions are employed into the regularization term and the fidelity term for enhancing the sensitivity to sharp edges and object boundaries. Secondly, the optimizing minimization method based on the alternating direction method of multipliers (ADMM) is proposed to solve the non-convex total variation optimization problem. The resulting subproblems either have closed-form solutions or can be solved by fast solvers, which makes the ADMM particularly efficient. In theory, with the help of the smoothing technique and Kurdyka-Lojasiewicz function, we prove that the sequence generated by the ADMM converges to a stationary point when the penalty parameter is above a computable threshold. The numerical experiments illustrate that our proposed non-convex total variation model outperforms the existing convex and non-convex total variation models.

Asynchronous dissipative control design for semi-Markovian jump systems with uncertain probability distribution functions of sojourn-time

This paper is concerned with the problem of asynchronous mode-dependent dissipative control for continuous-time semi-Markovian jump systems (S-MJSs), where the probability distribution functions of sojourn-time are established in a generic form to cover uncertainties that occur due to experimental limitations and errors. Further, based on a control-mode-dependent Lyapunov function, this paper proposes a robust control design method developed such that 1) the control system performance can be further improved compared to using system-mode-dependent Lyapunov functions, 2) the block size of stabilization conditions can be reduced, and 3) the non-convex terms combined with either positive or negative transition rates can be explicitly addressed. Finally, three illustrative examples are given to show the effectiveness of the proposed method.

Coordinated scheduling of integrated power and gas grids in consideration of gas flow dynamics

Natural gas, hydrogen, methane, or their mixture play important roles in the energy systems, and the coordinated optimal scheduling for the integrated energy systems are requisite in consideration of complicated system characteristics. This paper focuses on mixed-integer second-order cone programming-based optimal scheduling for the electric grids with gas flow dynamics in consideration of AC power flow, unit commitment, and line switching. For the gas grids, differential continuity equations and differential momentum equations, representing gas flow dynamics, are discretized to a group of algebraic equations with the implicit trapezoidal rules. In the algebraic equations, nonconvex bilinear terms with integer variables, representing gas flow directions, are transformed into linear inequality constraints with McCormick envelopes, and quadratic terms are relaxed by the second-order cone (SOC) approach. For the electric grids, SOC relaxation-based AC power flow models are employed, and the line switching problem is modeled by the improved SOC relaxation approach with McCormick envelopes. The entire problem is established as a mixed-integer second-order cone programming model, and the impacts of gas flow dynamics on the feasible region compared to the steady-state region are discussed.

A new approach for Cauchy noise removal

<abstract> In this paper, a new total generalized variational (TGV) model for restoring images with Cauchy noise is proposed, which contains a non-convex fidelity term and a TGV regularization term. In order to obtain a strictly convex model, we add an appropriate proximal term to the non-convex fidelity term. We prove that the solution of the proposed model exists and is unique. Due to the convexity of the proposed model and in order to get a convergent algorithm, we employ an alternating minimization algorithm to solve the proposed model. Finally, we demonstrate the performance of our scheme by numerical examples. Numerical results demonstrate that the proposed algorithm significantly outperforms some previous methods for Cauchy noise removal. </abstract>

Joint Mobile Charging and Coverage-Time Extension for Unmanned Aerial Vehicles

In modern networks, the use of drones as mobile base stations (MBSs) has been discussed for coverage flexibility. However, the realization of drone-based networks raises several issues. One of critical issues is drones are extremely power-hungry. To overcome this, we need to characterize a new type of drones, so-called charging drones, which can deliver energy to MBS drones. Motivated by the fact that the charging drones also need to be charged, we deploy ground-mounted charging towers for delivering energy to the charging drones. We introduce a new energy-efficiency maximization problem, which is partitioned into two independently separable tasks. More specifically, as our first optimization task, two-stage charging matching is proposed due to the inherent nature of our network model, where the first matching aims to schedule between charging towers and charging drones while the second matching solves the scheduling between charging drones and MBS drones. We analyze how to convert the formulation containing non-convex terms to another one only with convex terms. As our second optimization task, each MBS drone conducts energy-aware time-average transmit power allocation minimization subject to stability via Lyapunov optimization. Our solutions enable the MBS drones to extend their lifetimes; in turn, network coverage-time can be extended.

More relaxed stability analysis and positivity analysis for positive polynomial fuzzy systems via membership functions dependent method

In this paper, the conservatism source of the positivity and stability analysis results of positive polynomial fuzzy-model-based (PPFMB) control system are studied. Also, in order to improve the flexibility of controller design, a fuzzy controller that does not depend on the membership functions of the fuzzy model is designed. In the existing literatures, it is proved that the LCLF can reduce the conservatism of stability results. However, the LCLF generally results in non-convex conditions which is still a conservatism source. To handle the non-convex conditions, the sector nonlinear concept is applied to handle non-convex terms in stability conditions, and the obstacles caused by mismatched membership functions can be eliminated by PLMF dependent method. In addition, to relax the conservatism caused by the lack of membership functions information, the PLMF dependent positivity analysis are performed for the first time. Meanwhile, PLMF dependent method is extended to stability conditions to obtained more relaxed conditions. Finally, a simulation example is presented to verify the feasibility of this method.

A Proximal Iteratively Reweighted Approach for Efficient Network Sparsification

The huge size of deep neural networks makes it difficult to deploy on the embedded platforms with limited computation resources directly. In this article, we propose a novel trimming approach to determine the redundant parameters of the trained deep neural network in a layer-wise manner to produce a compact neural network. This is achieved by minimizing a nonconvex sparsity-inducing term of the network parameters while maintaining the response close to the original one. We present a proximal iteratively reweighted method to resolve the resulting nonconvex model, which approximates the nonconvex objective by a weighted l1 norm of the network parameters. Moreover, to alleviate the computational burden, we develop a novel termination criterion during the subproblem solution, significantly reducing the total pruning time. Global convergence analysis and a worst-case O(1/k) ergodic convergence rate for our proposed algorithm is established. Numerical experiments demonstrate the proposed approach is efficient and reliable.

Sharp formulations of nonconvex piecewise linear functions to solve the economic dispatch problem with valve-point effects

Wire drawing effects in a unit with several steam admission valves produce some ripples in the unit cost characteristic. Representing the ripples as sinusoidal terms gives rise to a nonconvex cost characteristic. This paper focuses on the solution of the economic dispatch (ED) problem with the nonconvex cost functions. To this end, this work presents a new framework in which, firstly, the complex problem is tightly transformed into a set of linear programming (LP) problems using an iterative piecewise linear function (PLF) approximation especially developed for the nonconvex terms. Subsequently, five strong models including convex combination, disaggregated convex combination, multiple choice, incremental, and logarithmic techniques are proposed to represent the set of built LP problems as a mixed-integer programming (MIP) problem which efficiently can be solved by a robust powerful algorithm. Then, the obtained solution from the MIP algorithm is used to warm-start an enhanced nonlinear programming (NLP) model and to further improve the solution. In extensive experiments, we compare the performance of the presented methods in terms of solution quality, computational efficiency, and consistency with other approaches in the literature of the nonconvex ED. The tight PLF approximation and the five sharp and generally locally ideal MIP formulation techniques in conjunction with the existing efficient MIP and NLP algorithms result in the strong solution framework which can find a global or a virtually global solution and surpass the state-of-the-art methods in the area of the nonconvex ED in all considered case studies.

New convergence results for the inexact variable metric forward–backward method

Forward–backward methods are valid tools to solve a variety of optimization problems where the objective function is the sum of a smooth, possibly nonconvex term plus a convex, possibly nonsmooth function. The corresponding iteration is built on two main ingredients: the computation of the gradient of the smooth part and the evaluation of the proximity (or resolvent) operator associated with the convex term. One of the main difficulties, from both implementation and theoretical point of view, arises when the proximity operator is computed in an inexact way. The aim of this paper is to provide new convergence results about forward–backward methods with inexact computation of the proximity operator, under the assumption that the objective function satisfies the Kurdyka–Łojasiewicz property. In particular, we adopt an inexactness criterion which can be implemented in practice, while preserving the main theoretical properties of the proximity operator. The main result is the proof of the convergence of the iterates generated by the forward–backward algorithm in Bonettini et al. (2017) to a stationary point. Convergence rate estimates are also provided. At the best of our knowledge, there exists no other inexact forward–backward algorithm with proved convergence in the nonconvex case and equipped with an explicit procedure to inexactly compute the proximity operator.

A generalized benders decomposition-based global optimization approach to symbolic regression for explicit surrogate modeling from limited data information

Solving noncovex Disjunctive Programming (DP) problems for representing symbolic regression trees to build explicit surrogate models from limited data information is challenging. This paper presents an effective global optimization approach to deal with these DPs. Piecewise McCormick envelope-based strategy is used to relax the nonconvex bilinear terms and nonconvex inequality constraints of the DP. A large-scale convex mixed integer nonlinear programing (MINLP) is then generated via the convex hull. A tailored linearization strategy further relaxes tightly all nonlinear terms in the convex MINLP to generate a mixed integer linear programming which is finally solved to global optimality by Generalized Benders Decomposition-based algorithm. Through testing three numerical simulations and Cetane number prediction, when compared to the monolithic approach of the state-of-the-art global optimization solvers such as BARON, the proposed approach is shown to reduce the solution time by up to two orders of magnitude and construct more accurate explicit surrogate models.

Improved RPCA method via non‐convex regularisation for image denoising

The traditional robust principal component analysis (RPCA) model is based on the nuclear norm, which usually underestimates the singular values of the low-rank matrix. As a consequence, the restoration image experiences serious interference by Gaussian noise, and the image quality degenerates during the denoising process. Therefore, an improved RPCA method via non-convex regularisation terms is proposed to remedy the above shortcomings. First, in order to estimate the singular value of the low-rank matrix more accurately, the authors employ the non-convex penalty function and add a weight vector to it. Then, the regularisation with non-convex penalty function and its weighted version are used to replace the nuclear norm and entry-wise norm in original RPCA, respectively, to establish an improved model. Finally, an optimal solution algorithm is derived by developing the alternating direction multiplier method. Experimental results show that the proposed method has better performance in terms of both quantitative measurement and visual perception quality than other several state-of-the-art image denoising methods.

A data-driven group-sparse feature extraction method for fault detection of wind turbine transmission system

Vibration monitoring using sensors mounted on machines is widely used for rotating machinery fault diagnosis. The periodic overlapping group sparsity (POGS) method has been developed in previous work of authors, and is an effective technique for detecting faults induced in rotating machines. However, the regularization parameter of the POGS problem is roughly specified via a look-up table provided in the original work. To address this problem, a data-driven diagnostic method, which is termed the adaptively regularized periodic overlapping group sparsity (ARPOGS), is proposed in this paper. The non-stationary fault feature ratio which is defined in the Hilbert domain is employed to guide the optimal regularization parameter. The criterion of setting the interval of candidate regularization parameters is also discussed. The ARPOGS is developed in terms of a convex optimization problem, while non-convex regularizations are used to further promote the sparsity. Since the non-convex penalty term is used and the whole objective function is constrained as a convex optimization problem, the sparsity of useful fault features is maximally induced. A simulated signal is formulated to verify the performance of the proposed method for periodic feature extraction. Finally, the effectiveness of the proposed ARPOGS method is validated by analyzing real data collected from a wind turbine transmission system. The results demonstrated that the proposed method can effectively and automatically extract periodic-group-sparse features from noisy vibration signals.

A nonconvex [formula omitted] model for image restoration with impulse noise

In this paper, we propose a model for image restoration with blur and impulse noise, it is composed of l1 data fitting term and a nonconvex regularization term, which is the weighted difference of l1-norm and l2-norm based on wavelet frame. The combined model is difficult to solve by the corresponding Euler Lagrangian equations, here we solve it by alternating direction method of multipliers (ADMM). We describe the detailed process of the algorithm, and establish the convergence of the algorithm. The experimental outcomes on different blurs and different impulse noises demonstrate that the proposed approach is better than those existing methods in terms of standard signal-to-noise ratio (PSNR), relative error (ReErr), and visual quality.

Global dynamic optimization using edge-concave underestimator

Optimization of problems with embedded system of ordinary differential equations (ODEs) is challenging and the difficulty is amplified due to the presence of nonconvexity. In this article, a deterministic global optimization method is presented for systems consisting of an objective function and constraints with integral terms and an embedded set of nonlinear parametric ODEs. The method is based on a branch-and-bound algorithm that uses a new class of underestimators recently proposed by Hasan (J Glob Optim 71:735–752, 2018). At each node of the branch-and-bound tree, instead of using a convex relaxation, an edge-concave underestimator or the linear facets of its convex envelope is used to compute a lower bound. The underestimator is constructed by finding valid upper bounds on the diagonal elements of the Hessian matrix of the nonconvex terms. Time dependent bounds on the state variables and diagonal elements of the Hessian are obtained by solving an auxiliary set of ODEs that is derived using the notion of differential inequalities. The performance of the edge-concave relaxation is compared to other approaches on several test problems.

Sparse multiple instance learning with non-convex penalty

Multiple instance learning (MIL) usually plays an important role in weakly labeled learning. It is able to reduce cost noted by accurate annotations and remove noisy samples with useless labels in large scale databases. However, in many real applications, the training data always contains many redundant features. In order to remove those residual features and improve the computational efficiency, some convex and nonconvex regularization terms are applied. In this paper we apply the MCP function to propose an efficient sparse modle SyMIL, and then use the proximal stochastic subgradient method (PSSG) to solve it. The convergence analysis of the algorithm is given in details. Extensive experiments show that our model can reduce redundant features effectively while ensuring the accuracy. Quantitative comparisons against several regularization terms demonstrate the advantages of the MCP regularizer in this model. Moreover, PSSG can be observed to have slightly faster convergence rate than some other methods.