Nonsmooth Objective Function Research Articles

Solution approach for optimal power flow considering wind turbine and environmental emissions

The power grid is changing expeditiously with the increasing penetration of renewable energy sources (RES). Optimal utilization of RES reduces the burden on the primary grid and makes the grid more resilient. Traditional optimal power flow (OPF) is a complex problem in power management systems, and the complexity further increases with the integration of RES due to their intermittency. This paper presents the complete formulation of the OPF model incorporating wind turbines (WT) and environmental emissions for proper scheduling, planning, and efficient operation of thermal generating units (TGU) using the Ant Lion Optimization (ALO) algorithm. The formulation of the OPF problem comprises forecasted active power generation of WT, depending on the real-time measurement and probabilistic wind speed models. The results are analyzed from the perspective of operating cost, voltage profile, and transmission power losses in the system. The OPF approach and the solution methodology are tested on the IEEE 30 and IEEE 57-bus systems. The effectiveness of the proposed ALO algorithm is evaluated against well-established algorithms like Particle Swarm Optimization and Teaching-learning-based optimization. The comparison emphasizes the effectiveness of the ALO approach for solving various OPF problems with complex and non-smooth objective functions.

Asymptotics for M-type smoothing splines with non-smooth objective functions

M-type smoothing splines are a broad class of spline estimators that include the popular least-squares smoothing spline but also spline estimators that are less susceptible to outlying observations and model misspecification. However, available asymptotic theory only covers smoothing spline estimators based on smooth objective functions and consequently leaves out frequently used resistant estimators such as quantile and Huber-type smoothing splines. We provide a general treatment in this paper and, assuming only the convexity of the objective function, show that the least-squares (super-)convergence rates can be extended to M-type estimators whose asymptotic properties have not been hitherto described. We further show that auxiliary scale estimates may be handled under significantly weaker assumptions than those found in the literature and we establish optimal rates of convergence for the derivatives, which have not been obtained outside the least-squares framework. A simulation study and a real-data example illustrate the competitive performance of non-smooth M-type splines in relation to the least-squares spline on regular data and their superior performance on data that contain anomalies.

Sparsity reconstruction using nonconvex TGpV-shearlet regularization and constrained projection

In many sparsity-based image processing problems, compared with the convex ℓ1 norm approximation of the nonconvex ℓ0 quasi-norm, one can often preserve the structures better by taking full advantage of the nonconvex ℓp quasi-norm (0≤p<1). In this paper, we propose a nonconvex ℓp quasi-norm approximation in the total generalized variation (TGV)-shearlet regularization for image reconstruction. By introducing some auxiliary variables, the nonconvex nonsmooth objective function can be solved by an efficient alternating direction method of multipliers with convergence analysis. Especially, we use a generalized iterated shrinkage operator to deal with the ℓp quasi-norm subproblem, which is easy to implement. Extensive experimental results show clearly that the proposed nonconvex sparsity approximation outperforms some state-of-the-art algorithms in both the visual and quantitative measures for different sampling ratios and noise levels.

Calibration-Less Multi-Coil Compressed Sensing Magnetic Resonance Image Reconstruction Based on OSCAR Regularization.

Over the last decade, the combination of compressed sensing (CS) with acquisition over multiple receiver coils in magnetic resonance imaging (MRI) has allowed the emergence of faster scans while maintaining a good signal-to-noise ratio (SNR). Self-calibrating techniques, such as ESPiRIT, have become the standard approach to estimating the coil sensitivity maps prior to the reconstruction stage. In this work, we proceed differently and introduce a new calibration-less multi-coil CS reconstruction method. Calibration-less techniques no longer require the prior extraction of sensitivity maps to perform multi-coil image reconstruction but usually alternate estimation sensitivity map estimation and image reconstruction. Here, to get rid of the nonconvexity of the latter approach we reconstruct as many MR images as the number of coils. To compensate for the ill-posedness of this inverse problem, we leverage structured sparsity of the multi-coil images in a wavelet transform domain while adapting to variations in SNR across coils owing to the OSCAR (octagonal shrinkage and clustering algorithm for regression) regularization. Coil-specific complex-valued MR images are thus obtained by minimizing a convex but nonsmooth objective function using the proximal primal-dual Condat-Vù algorithm. Comparison and validation on retrospective Cartesian and non-Cartesian studies based on the Brain fastMRI data set demonstrate that the proposed reconstruction method outperforms the state-of-the-art (-ESPIRiT, calibration-less AC-LORAKS and CaLM methods) significantly on magnitude images for the T1 and FLAIR contrasts. Additionally, further validation operated on 8 to 20-fold prospectively accelerated high-resolution ex vivo human brain MRI data collected at 7 Tesla confirms the retrospective results. Overall, OSCAR-based regularization preserves phase information more accurately (both visually and quantitatively) compared to other approaches, an asset that can only be assessed on real prospective experiments.

Mixed integer nonlinear optimization models for the Euclidean Steiner tree problem in $$\mathbb {R}^d$$

New mixed integer nonlinear optimization models for the Euclidean Steiner tree problem in d-space (with \(d\ge 3\)) will be presented in this work. All models feature a nonsmooth objective function but the continuous relaxations of their set of feasible solutions are convex. From these models, four convex mixed integer linear and nonlinear relaxations will be considered. Each relaxation has the same set of feasible solutions as the set of feasible solutions of the model from which it is derived. Finally, preliminary computational results highlighting the main features of the presented relaxations will be discussed.

Distributed Nonsmooth Convex Optimization over Markovian Switching Random Networks with Two Step-Sizes

This paper investigates the distributed convex optimization problem over a multi-agent system with Markovian switching communication networks. The objective function is the sum of each agent’s local nonsmooth objective function, which cannot be known by other agents. The communication network is assumed to switch over a set of weight-balanced directed graphs with a Markovian property. The authors propose a consensus sub-gradient algorithm with two time-scale step-sizes to handle the Markovian switching topologies and the absence of global gradient information. With proper selection of step-sizes, the authors prove the almost sure convergence of all agents’ local estimates to the same optimal solution when the union graph of the Markovian network’ states is strongly connected and the Markovian chain is irreducible. The convergence rate analysis is also given for specific cases. Simulations are given to demonstrate the results.

A Dynamic Discriminative Canonical Correlation Analysis via Adaptive Weight Scheme

In multi-view learning, massive literature is devoted to exploring the intrinsic structure between cross-views. It is well known that canonical correlation analysis (CCA) is a conventional multi-view learning method, which considers the correlation between two views. However, it fails to utilize class information and is difficult to suit different issues to extract discriminative features. In this paper, we propose a novel cross-view discriminative feature learning method called dynamic discriminative canonical correlation analysis, which captures class information to yield discriminative features. More specifically, we develop an adaptive weight scheme of cross-view within-class and between-class scatters to make full use of distribution class information. In addition, an iterative algorithm with Cauchy inequalities and the Lagrange multiplier is proposed to handle the non-smooth objective function. Our method is applied to face recognition and multi-linguistic text classification tasks. Extensive experimental results reveal that the adaptive weight scheme plays a beneficial role and our method is an effective feature learning.

A Bfgs Method for the Problem of Building S-Shaped Curve

We consider the application of the BFGS method (Broyden, Fletcher, Goldfarb, Shanno) and its projective version L-BFGS-B for minimization of nonlinear function, which corresponds to finding solutions to a system of five nonlinear equations. Among them three equations are integral and depend on unknown parameters of integrands and unknown upper bounds for definite integrals. This system represents the problem of building an S-shaped curve in natural parameterization, which passes through the two given points with given tangents inclination angles and provides a given tangent inclination angle at an intermediate point with a given abscissa. The tangent inclination angle at the point with known abscissa is used to control the inflection point of the S-shaped curve, by which a fragment of the external contour of the Frankl nozzle is modeled.The material of the article is presented in three sections. Section 1 describes a system of five nonlinear equations and analyses its properties related to the existence of many solutions. In section 2 the optimization problem with a non-smooth objective function and linear constraints is presented, the global minima of which correspond to solutions of a system of nonlinear equations. There also is described the application of the modification of r-algorithm to find global minima of the optimization problem.Section 3 describes the application of BFGS and L-BFGS-B methods to find local minima of smooth functions corresponding to finding solutions of a system of nonlinear equations. We formulate two optimization problems with smooth objective functions and two-sided constraints on variables and describe the results of computational experiments for finding solution of the system of nonlinear equations, presented in Section 1, by S-shaped curve. It is shown that BFGS and L-BFGS-B methods are effective if the starting point is selected in such a neighborhood of the minimum point, where the minimized function is approximated fairly accurately by a convex quadratic function.Manuscript received 15.06.2020

Diminishing stepsize methods for nonconvex composite problems via ghost penalties: from the general to the convex regular constrained case

ABSTRACT In this paper, we first extend the diminishing stepsize method for nonconvex constrained problems presented in F. Facchinei, V. Kungurtsev, L. Lampariello and G. Scutari [Ghost penalties in nonconvex constrained optimization: Diminishing stepsizes and iteration complexity, To appear on Math. Oper. Res. 2020. Available at https://arxiv.org/abs/1709.03384.] to deal with equality constraints and a nonsmooth objective function of composite type. We then consider the particular case in which the constraints are convex and satisfy a standard constraint qualification and show that in this setting the algorithm can be considerably simplified, reducing the computational burden of each iteration.

Newton projection method as applied to assembly simulation

In this paper, we consider Newton projection method for solving the quadratic programming problem that emerges in simulation of joining process for assembly with compliant parts. This particular class of problems has specific features such as an ill-conditioned Hessian and a sparse matrix of constraints as well as a requirement for the large-scale computations. We use the projected Newton method with a quadratic rate of convergence and suggest some improvements to reduce the solving time: a method for solving the system of linear equations, so-called constraint recalculation method, and compare different approaches for step-size selection. We use the duality principle to formulate alternative forms of the minimization problem that, as a rule, can be solved faster. We describe how to solve the considered nonlinear minimization problem with the nonsmooth objective function by modifying Newton projection method and employing subgradients. In addition, we prove the convergence of the suggested algorithm. Finally, we compare Newton projection method with the other quadratic programming techniques on a number of assembly simulation problems.

Acceleration techniques for level bundle methods in weakly smooth convex constrained optimization

We develop a unified level-bundle method, called accelerated constrained level-bundle (ACLB) algorithm, for solving constrained convex optimization problems. where the objective and constraint functions can be nonsmooth, weakly smooth, and/or smooth. ACLB employs Nesterov’s accelerated gradient technique, and hence retains the iteration complexity as that of existing bundle-type methods if the objective or one of the constraint functions is nonsmooth. More importantly, ACLB can significantly reduce iteration complexity when the objective and all constraints are (weakly) smooth. In addition, if the objective contains a nonsmooth component which can be written as a specific form of maximum, we show that the iteration complexity of this component can be much lower than that for general nonsmooth objective function. Numerical results demonstrate the effectiveness of the proposed algorithm.

A Novel Method for Asynchronous Time-of-Arrival-Based Source Localization: Algorithms, Performance and Complexity.

In time-of-arrival (TOA)-based source localization, accurate positioning can be achieved only when the correct signal propagation time between the source and the sensors is obtained. In practice, a clock error usually exists between the nodes causing the source and sensors to often be in an asynchronous state. This leads to the asynchronous source localization problem which is then formulated to a least square problem with nonconvex and nonsmooth objective function. The state-of-the-art algorithms need to relax the original problem to convex programming, such as semidefinite programming (SDP), which results in performance loss. In this paper, unlike the existing approaches, we propose a proximal alternating minimization positioning (PAMP) method, which minimizes the original function without relaxation. Utilizing the biconvex property of original asynchronous problem, the method divides it into two subproblems: the clock offset subproblem and the synchronous source localization subproblem. For the former we derive a global solution, whereas the later is solved by a proposed efficient subgradient algorithm extended from the simulated annealing-based Barzilai–Borwein algorithm. The proposed method obtains preferable localization performance with lower computational complexity. The convergence of our method in Lyapunov framework is also established. Simulation results demonstrate that the performance of PAMP method can be close to the optimality benchmark of Cramér–Rao Lower Bound.

Approximate-Karush-Kuhn-Tucker Conditions and Interval Valued Vector Variational Inequalities

This Article deals with the Approximate Karush-Kuhn-Tucker (AKKT) optimality conditions for interval valued multiobjective function as a generalization of Karush-Kuhn-Tucker optimality conditions. Further, we establish relationship between vector variational inequality problems and multiobjective interval valued optimization problems under the assumption of LU−convex smooth and nonsmooth objective functions.

An accelerated proximal gradient algorithm for source-independent waveform inversion

Full Waveform Inversion (FWI) is a promising method for reconstructing subsurface parameters from the full information content in the seismogram. However, FWI is facing many difficulties in practice. The lack of the source wavelet information, the nonlinearity of the forward problems, and ill-posedness of the formulation all contribute to a pressing need for novel regularization techniques and fast algorithms to speed up and improve inversion results. In this paper, by making use of the variable projection method for nonlinear least squares problems, we develop a source-independent frequency-domain FWI strategy. Specifically, the source wavelet for each frequency is removed automatically using a minimum norm solution between the measured and simulated data. Therefore, the inversion process becomes source independent. In order to overcome the defects of overly smoothed edges caused by the classical Tikhonov regularization, sparsity constrained regularization is applied to FWI based on the ability of curvelets to efficiently represent geophysical images. However, non-differentiability of the objective function make it challenging to find an efficient numerical solution. By using the proximal mapping and the Barzilai-Borwein step size rule, we derive a new accelerated proximal gradient algorithm to handle such non-smooth objective functions. The numerical examples show that very good reconstruction results can be obtained by the proposed algorithm without knowledge of the source. Compared to the gradient descent method with a constant step size, the accelerated algorithm gaves a superior result with much higher resolution.

Combined heat and power system intelligent economic dispatch: A deep reinforcement learning approach

This paper proposed a Deep Reinforcement learning (DRL) approach for Combined Heat and Power (CHP) system economic dispatch which obtain adaptability for different operating scenarios and significantly decrease the computational complexity without affecting accuracy. In the respect of problem description, a vast of Combined Heat and Power (CHP) economic dispatch problems are modeled as a high-dimensional and non-smooth objective function with a large number of non-linear constraints for which powerful optimization algorithms and considerable time are required to solve it. In order to reduce the solution time, most engineering applications choose to linearize the optimization target and devices model. To avoid complicated linearization process, this paper models CHP economic dispatch problems as Markov Decision Process (MDP) that making the model highly encapsulated to preserve the input and output characteristics of various devices. Furthermore, we improve an advanced deep reinforcement learning algorithm: distributed proximal policy optimization (DPPO), to make it applicable to CHP economic dispatch problem. Based on this algorithm, the agent will be trained to explore optimal dispatch strategies for different operation scenarios and respond to system emergencies efficiently. In the utility phase, the trained agent will generate optimal control strategy in real time based on current system state. Compared with existing optimization methods, advantages of DRL methods are mainly reflected in the following three aspects: 1) Adaptability: under the premise of the same network topology, the trained agent can handle the economic scheduling problem in various operating scenarios without recalculation. 2) High encapsulation: The user only needs to input the operating state to get the control strategy, while the optimization algorithm needs to re-write the constraints and other formulas for different situations. 3) Time scale flexibility: It can be applied to both the day-ahead optimized scheduling and the real-time control. The proposed method is applied to two test system with different characteristics. The results demonstrate that the DRL method could handle with varieties of operating situations while get better optimization performance than most of other algorithms.

Group Guided Fused Laplacian Sparse Group Lasso for Modeling Alzheimer’s Disease Progression

As the largest cause of dementia, Alzheimer's disease (AD) has brought serious burdens to patients and their families, mostly in the financial, psychological, and emotional aspects. In order to assess the progression of AD and develop new treatment methods for the disease, it is essential to infer the trajectories of patients' cognitive performance over time to identify biomarkers that connect the patterns of brain atrophy and AD progression. In this article, a structured regularized regression approach termed group guided fused Laplacian sparse group Lasso (GFL-SGL) is proposed to infer disease progression by considering multiple prediction of the same cognitive scores at different time points (longitudinal analysis). The proposed GFL-SGL simultaneously exploits the interrelated structures within the MRI features and among the tasks with sparse group Lasso (SGL) norm and presents a novel group guided fused Laplacian (GFL) regularization. This combination effectively incorporates both the relatedness among multiple longitudinal time points with a general weighted (undirected) dependency graphs and useful inherent group structure in features. Furthermore, an alternating direction method of multipliers- (ADMM-) based algorithm is also derived to optimize the nonsmooth objective function of the proposed approach. Experiments on the dataset from Alzheimer's Disease Neuroimaging Initiative (ADNI) show that the proposed GFL-SGL outperformed some other state-of-the-art algorithms and effectively fused the multimodality data. The compact sets of cognition-relevant imaging biomarkers identified by our approach are consistent with the results of clinical studies.

VR-SGD: A Simple Stochastic Variance Reduction Method for Machine Learning

In this paper, we propose a simple variant of the original SVRG, called variance reduced stochastic gradient descent (VR-SGD). Unlike the choices of snapshot and starting points in SVRG and its proximal variant, Prox-SVRG, the two vectors of VR-SGD are set to the average and last iterate of the previous epoch, respectively. The settings allow us to use much larger learning rates, and also make our convergence analysis more challenging. We also design two different update rules for smooth and non-smooth objective functions, respectively, which means that VR-SGD can tackle non-smooth and/or non-strongly convex problems directly without any reduction techniques. Moreover, we analyze the convergence properties of VR-SGD for strongly convex problems, which show that VR-SGD attains linear convergence. Different from most algorithms that have no convergence guarantees for non-strongly convex problems, we also provide the convergence guarantees of VR-SGD for this case, and empirically verify that VR-SGD with varying learning rates achieves similar performance to its momentum accelerated variant that has the optimal convergence rate $\mathcal {O}(1/T^2)$ O ( 1 / T 2 ) . Finally, we apply VR-SGD to solve various machine learning problems, such as convex and non-convex empirical risk minimization, and leading eigenvalue computation. Experimental results show that VR-SGD converges significantly faster than SVRG and Prox-SVRG, and usually outperforms state-of-the-art accelerated methods, e.g., Katyusha.

Data-Driven Nonsmooth Optimization

In this work, we consider methods for solving large-scale optimization problems with a possibly nonsmooth objective function. The key idea is to first parametrize a class of optimization methods using a generic iterative scheme involving only linear operations and applications of proximal operators. This scheme contains some modern primal-dual first-order algorithms like the Douglas--Rachford and hybrid gradient methods as special cases. Moreover, we show weak convergence of the iterates to an optimal point for a new method which also belongs to this class. Next, we interpret the generic scheme as a neural network and use unsupervised training to learn the best set of parameters for a specific class of objective functions while imposing a fixed number of iterations. In contrast to other approaches of “learning to optimize," we present an approach which learns parameters only in the set of convergent schemes. Finally, we illustrate the approach on optimization problems arising in tomographic reconstruction and image deconvolution, and train optimization algorithms for optimal performance given a fixed number of iterations.

Penalty Dual Decomposition Method for Nonsmooth Nonconvex Optimization—Part II: Applications

In Part I of this paper, we proposed and analyzed a novel algorithmic framework, termed penalty dual decomposition (PDD), for the minimization of a nonconvex nonsmooth objective function, subject to difficult coupling constraints. Part II of this paper is devoted to evaluation of the proposed methods in the following three timely applications, ranging from communication networks to data analytics: i) the max-min rate fair multicast beamforming problem; ii) the sum-rate maximization problem in multi-antenna relay broadcast networks; and iii) the volume-min based structured matrix factorization problem. By exploiting the structure of the aforementioned problems, we show that effective algorithms for all these problems can be devised under the PDD framework. Unlike the state-of-the-art algorithms, the PDD-based algorithms are proven to achieve convergence to stationary solutions of the aforementioned nonconvex problems. Numerical results validate the efficacy of the proposed schemes.

ON THE PROBLEMS OF BILEVEL OPTIMIZATION UNDER RCPLD CONSTRAINT QUALIFICATIONS

Multilevel optimization problems often arise in various applications (in economics, ecology, power engineering and other areas) when modeling complex systems with a hierarchical structure associated with independent actions of subsystems. The difficulty of analyzing such complex systems requires first of all the study of bilevel models, the management of which would be an integral part of the analysis of more complex systems. In solving bilevel programming problems, an important role is played by the property of partial calmness, the presence of which allows us to reduce the bilevel problem to the classical nonlinear programming problem with a nonsmooth objective function. It is known that linear bilevel programming problems are partially stable. The proof of this property for more complex problems meets difficulties. In particular, our article shows the inaccuracy of some results in this area. The goal of the paper is to obtain some new results in the partial calmness of bilevel programming. In particular, new sufficient conditions for bilevel problems are proved. The results are obtained on the base of Lipschitz-like properties for multivalued mappings. In the paper we propose new sufficient conditions for partial calmness which are based on some modification of the known constraint qualification RCPLD which have been proposed by the researches Andreani, Haeser, Schuverdt and Silva.